Web Search

Web Search

Run a search across one or many providers, get a JSON file you can pipe through rg, jq, or any other tool. Use this whenever the answer depends on current information that may not be in your training data — recent releases, today's news, a specific vendor's API as of this week, real-world example usage of a library, security advisories, etc.

Decision: should I use this skill?

If you decide to search, choose the right invocation pattern below.

How to invoke

Three patterns, in order of complexity:

Pattern 1: single provider (most common)

python3 scripts/web-search "<your query>"

Uses the configured default provider (DuckDuckGo if no API keys are set up). Reads stdout for a human-readable preview AND a machine footer that names the result file:

=== duckduckgo (10 results, 393ms) ===
  [1] Python (programming language)
      https://en.wikipedia.org/wiki/Python_(programming_language)
      Python is a high-level, general-purpose programming language...
  [2] ...

--- WEBSEARCH ---
RESULT_FILE: /tmp/web-search/20260501T073731Z/combined.json
RUN_DIR:     /tmp/web-search/20260501T073731Z
PROVIDERS:   duckduckgo
TOTAL_RESULTS: 10

Pattern 2: specific provider

python3 scripts/web-search --provider tavily "<query>"
python3 scripts/web-search --provider perplexity "<query>" --max-results 5
python3 scripts/web-search --provider anthropic "<query>"

Use a specific provider when you want a known feature — Perplexity for recency-filtered queries, Tavily for clean LLM snippets, Anthropic/OpenAI/xAI for model-mediated synthesis.

Pattern 3: parallel multi-provider

# Subset of providers in parallel:
python3 scripts/web-search --providers tavily,brave,perplexity "<query>"

# Every configured provider in parallel:
python3 scripts/web-search --all "<query>"

Use multi-provider when:

• the topic is novel and you want diverse indexes triangulating
• a single provider returned thin or unrelated results on the first try
• the user asked for "thorough research" or "from multiple sources"

The skill saves each provider's output independently, so partial failures still produce useful data.

Pattern 4: sequential fallback chain (config-driven)

Add a fallback field to your config and the skill runs providers sequentially, stopping at the first one that returns results:

{
  "default": "brave",
  "fallback": ["openai", "duckduckgo", "mwmbl"]
}

Now python3 scripts/web-search "<query>" (no --provider flag) tries brave first; if it errors out OR returns 0 results, it falls through to openai; missing credentials silently skip a slot, so the chain proceeds to duckduckgo, etc. Footer reports MODE: fallback and FALLBACK_TRIED: <providers that failed>.

Use a fallback chain when:

• you want resilience against rate limits / outages on a paid provider
• you want a free safety net under a paid primary
• you're future-proofing — list openai in the chain today and it activates the moment you add an apiKey

--provider, --providers, --all all override the chain — use those when you want a specific behavior for one call.

Reading the results

Every run writes to <TMP_ROOT>/<run-id>/. The path appears in stdout as RESULT_FILE:. Default <TMP_ROOT> is /tmp/web-search on macOS/Linux and <system-temp>/web-search on Windows.

Three useful flags for downstream automation:

Pipelines (the point of this skill)

The whole point of dumping JSON to disk is so you can filter, transform, and combine results without re-running the search. Examples:

# Get every URL across all providers:
RESULT=$(python3 scripts/web-search --all --print-path-only "...")
jq -r '.manifests[].results[].url' "$RESULT"

# Filter results to lines mentioning "release":
rg -i "release" "$RESULT"

# Stream straight from --json without disk:
python3 scripts/web-search --json "..." | jq '.manifests[].results[0]'

# Top result per provider:
jq -r '.manifests[] | "\(.provider): \(.results[0].title) - \(.results[0].url)"' "$RESULT"

# Dedupe across providers, sorted by score:
jq '[.manifests[].results[]] | unique_by(.url) | sort_by(-(.score // 0)) | .[:10]' "$RESULT"

Many more recipes in `references/pipelines.md`.

When the wrapper is not available

If the host has curl but no Python (some Windows machines, distroless containers, jailed CI), invoke the providers directly. Every provider's exact request shape is documented in `references/curl-recipes.md`. The output schema matches what the skill produces internally — same fields, same names.

Configuration

The skill works with zero configuration — DuckDuckGo and MWMBL accept anonymous calls. Provider variety improves with credentials. Drop a config file in any of these locations:

Or set the matching env var (TAVILY_API_KEY, BRAVE_API_KEY, etc.) and the skill picks it up. See `references/setup.md` for per-provider key acquisition steps.

Validate your setup:

python3 scripts/web-search --check
python3 scripts/web-search --list-providers

baseUrl override (every provider)

Every provider supports a baseUrl field that fully replaces the default endpoint:

{
  "providers": {
    "tavily":    {"apiKey": "tvly-...", "baseUrl": "https://eu.api.tavily.com/search"},
    "anthropic": {"apiKey": "sk-ant-...", "baseUrl": "https://anthropic-gw.internal.example.com/v1/messages"},
    "openai":    {"apiKey": "sk-...", "baseUrl": "http://localhost:8787/openai/v1/responses"}
  }
}

Use it for self-hosted proxies, regional mirrors, corporate gateways, or local mocks. The override is complete — the skill does not concatenate with the default. See `references/base-urls.md` for concrete patterns.

Cross-platform behavior

Identical commands work on macOS, Linux, WSL, Git Bash, Windows native, and containers. The Python script uses only the standard library and pathlib.Path for path handling — no shell-isms, no /-vs-\ issues.

Windows-specific notes:

# PowerShell:
$env:TAVILY_API_KEY = "tvly-..."
python scripts\web-search "your query"

# cmd.exe:
set TAVILY_API_KEY=tvly-...
python scripts\web-search "your query"

The shebang #!/usr/bin/env python3 is ignored on Windows — invoke python (or py) explicitly. Full guide in `references/platform-notes.md`.

Provider selection guide

Full catalog with API shapes, server caps, and quirks: `references/providers/index.md`. Each provider has its own dedicated page (e.g. `providers/tavily.md`, `providers/anthropic.md`).

Domain filtering

Restrict results to or away from specific domains:

python3 scripts/web-search "..." --include docs.python.org,python.org
python3 scripts/web-search "..." --exclude reddit.com,medium.com

Both --include and --exclude accept comma-separated lists. Some providers (z-ai, xAI) have stricter caps — see the per-provider pages under `references/providers/`. Allowed and blocked are mutually exclusive at call time.

Cost awareness

Model-mediated providers (xai, openai, anthropic) charge per call AND per token. Set maxUses (Anthropic) or max_tool_calls (OpenAI) in config to cap runaway agentic loops. For broad sweeps prefer the cheaper non-model providers and reserve LLM-mediated search for synthesis.

Failure modes

The raw upstream response is always saved at <run-dir>/<provider>-raw.json so you can inspect failures without re-querying.

Reference index

Quick reference

python3 scripts/web-search [OPTIONS] "<query>"

  --provider NAME         Single provider (overrides default)
  --providers a,b,c       Comma-separated subset, run in parallel
  --all                   Run every configured provider in parallel
  --max-results N         Cap per provider (default 10)
  --include DOMAIN[,...]  Allow-list domains
  --exclude DOMAIN[,...]  Block-list domains
  --config PATH           Override config file
  --tmp-dir PATH          Override output directory
  --json                  Print full manifest to stdout
  --print-path-only       Print only the result file path
  --raw                   Print raw upstream JSON
  --list-providers        Show configured providers and exit
  --check                 Validate config, exit 0 if any provider works

When in doubt:

python3 scripts/web-search --provider duckduckgo "<query>"   # always works, no key needed

web-search

LLM-neutral web search skill for AI coding agents (pi, hermes-agent, openclaw, opencode, claude-code, etc.).

Instead of hard-coding a single search provider into every agent, this skill lets the agent invoke a single command (scripts/web-search) that fans out across 12 search providers, dumps raw + normalized results to a temp directory, and prints a path the agent can pipe through rg, jq, or any other tool.

Zero config out of the box — DuckDuckGo and MWMBL accept anonymous calls. Add API keys for richer providers (Tavily, Brave, Perplexity, Anthropic, OpenAI, etc.) when you want them.

Why a skill, not a library?

Agent harnesses already have a way to load skills on demand. Embedding web search as a skill means:

• No fixed context cost — the skill loads only when the agent needs to search.
• No vendor lock-in — swap providers per call, or fan out across many in parallel.
• No SDK churn — pure Python 3 stdlib + curl. Works wherever Python 3 runs.
• Pipeline-native — results live as JSON files on disk so rg, jq, grep, python -c all just work.

Supported providers (12)

Every provider supports a `baseUrl` override for self-hosted proxies, regional mirrors, corporate gateways, or local mocks. See `references/base-urls.md`.

Full provider catalog with endpoints, request/response shapes, and quirks: `references/providers/index.md`. Each provider has its own dedicated reference page with API spec, setup, curl recipe, and quirks all in one place.

Setup instructions for each provider's API key: `references/setup.md`.

Quick start

Zero-config (no API key)

python3 scripts/web-search "what is the latest python version"

Add API keys

mkdir -p ~/.config/web-search                                   # macOS / Linux
cp config/config.example.json ~/.config/web-search/config.json
$EDITOR ~/.config/web-search/config.json

On Windows, drop the file in %APPDATA%\web-search\config.json instead.

Common invocations

# Specific provider:
python3 scripts/web-search --provider tavily "latest python version"

# Parallel across multiple providers:
python3 scripts/web-search --providers tavily,brave,perplexity "..."

# Every configured provider in parallel:
python3 scripts/web-search --all "..."

# Pipeline-friendly:
python3 scripts/web-search "kubernetes 1.31 release notes" | rg -i "release|stable"
jq '.manifests[].results[].url' "$(python3 scripts/web-search --print-path-only "...")"

See `SKILL.md` for the full agent-facing documentation.

Cross-platform

Runs on macOS, Linux, WSL, Git Bash, native Windows, and any container with Python 3.8+. Pure stdlib — no pip install, no Node, no jq required to use the wrapper. (jq and rg are recommended for downstream filtering.) Full per-OS notes in `references/platform-notes.md`.

Layout

web-search/
├── SKILL.md                      # Agent-facing skill instructions (LLM-neutral)
├── README.md                     # Human-facing project overview (this file)
├── LICENSE
├── scripts/
│   └── web-search                # Single-file Python 3 entry point (stdlib only)
├── references/
│   ├── providers/                # One file per provider — full API spec, setup, curl, quirks
│   │   ├── index.md              # Comparison table, selection guide, "adding a provider"
│   │   ├── duckduckgo.md         # Free, no auth — Instant Answer API
│   │   ├── mwmbl.md              # Free, no auth — open-source independent index
│   │   ├── exa.md                # Optional key — semantic search
│   │   ├── tavily.md             # Required — LLM-friendly clean snippets
│   │   ├── brave.md              # Required — independent index
│   │   ├── serper.md             # Required — Google's organic via paid proxy
│   │   ├── google-cse.md         # Required + CSE id — official Google
│   │   ├── z-ai.md               # Required — Chinese-language coverage
│   │   ├── perplexity.md         # Required — pure /search OR sonar
│   │   ├── xai.md                # Required + model — Grok web_search + x_search
│   │   ├── openai.md             # Required + model — Responses API + web_search
│   │   └── anthropic.md          # Required + model — Messages API + web_search
│   ├── setup.md                  # Per-provider API key acquisition guides (cross-cutting)
│   ├── pipelines.md              # rg/jq/grep filtering recipes
│   ├── multi-provider.md         # Parallel search strategies + consensus ranking
│   ├── curl-recipes.md           # All curl one-liners in one place (Python-less fallback)
│   ├── result-schema.md          # Canonical normalized output format
│   ├── platform-notes.md         # macOS / Linux / Windows / WSL / containers
│   └── base-urls.md              # baseUrl override patterns (proxies, gateways, mocks)
└── config/
    └── config.example.json       # Configuration template (every provider documented)

License

MIT. See LICENSE.

Base URL Overrides

Every provider supports a baseUrl field that fully replaces the default endpoint. Set it once in config to point at a self-hosted proxy, regional mirror, corporate gateway, or local mock — without forking the skill or rebuilding anything.

How it works

When the skill builds a request for any provider, it resolves the endpoint in this exact order:

1. providers.<name>.baseUrl from config (if non-empty string) 2. DEFAULT_ENDPOINTS[<name>] (the upstream-official URL)

The override is complete — including the path. No prefix concatenation. No string formatting. Whatever you set is what gets called. This means you control:

• the scheme (https → http for local mocks)
• the hostname (regional mirror, corporate gateway, internal cluster IP)
• the port (test servers, proxies)
• the path (/v1/search → /api/v1/web_search if your proxy renamed routes)
• pre-baked query parameters (rare, but supported — parse_qsl preserves them)

Where to put it

{
  "providers": {
    "tavily": {
      "apiKey": "tvly-...",
      "baseUrl": "https://eu.api.tavily.com/search"
    },
    "anthropic": {
      "apiKey": "sk-ant-...",
      "baseUrl": "https://anthropic-gw.internal.example.com/v1/messages"
    }
  }
}

You can mix-and-match — override one provider's URL while leaving the rest at default.

Common patterns

1. Self-hosted proxy or gateway

You operate a corporate proxy that injects auth, rate-limits, or audit-logs every outbound API call:

{
  "providers": {
    "openai":    {"apiKey": "sk-...", "baseUrl": "https://llm-gw.example.internal/openai/v1/responses"},
    "anthropic": {"apiKey": "sk-ant-...", "baseUrl": "https://llm-gw.example.internal/anthropic/v1/messages"},
    "tavily":    {"apiKey": "tvly-...", "baseUrl": "https://search-gw.example.internal/tavily/search"}
  }
}

The skill sends the same request body and headers — the gateway is responsible for forwarding upstream.

2. Regional mirrors (latency / data residency)

When a vendor publishes a region-specific endpoint:

{
  "providers": {
    "tavily": {
      "apiKey": "tvly-...",
      "baseUrl": "https://api-eu.tavily.com/search"
    }
  }
}

Useful for EU residency requirements, latency-sensitive workloads, or vendor-published regional failover.

3. Local mocks for tests

A local HTTP server returns canned JSON for fast unit tests:

{
  "providers": {
    "exa":    {"baseUrl": "http://localhost:8787/exa/search"},
    "tavily": {"apiKey": "test", "baseUrl": "http://localhost:8787/tavily/search"}
  }
}

Spin up the mock with any tool (Python http.server, Node msw, Go chi, etc.) and hit it without changing the test command.

Example mock server in 20 lines of Python:

import http.server, json
class H(http.server.BaseHTTPRequestHandler):
    def do_POST(self):
        self.send_response(200); self.end_headers()
        self.wfile.write(json.dumps({
            "results": [{"title": "Mock", "url": "https://example.test", "content": "mock snippet"}]
        }).encode())
http.server.HTTPServer(("127.0.0.1", 8787), H).serve_forever()

4. Anthropic / OpenAI via cloud gateways

Anthropic on Bedrock, Vertex, or Azure all expose Anthropic-compatible endpoints:

{
  "providers": {
    "anthropic": {
      "apiKey": "<gateway-token>",
      "baseUrl": "https://anthropic-gw.bedrock.us-east-1.amazonaws.com/v1/messages"
    }
  }
}

The skill's request body is the standard Anthropic Messages API shape — gateways that speak Anthropic's wire format work transparently.

For Azure-hosted OpenAI, you typically need to switch the path format slightly:

{
  "providers": {
    "openai": {
      "apiKey": "<azure-key>",
      "baseUrl": "https://my-azure.openai.azure.com/openai/v1/responses?api-version=2026-04-01"
    }
  }
}

(The skill preserves any pre-existing query parameters on the override URL.)

5. Vendor-staged variants

Many vendors publish staging or beta endpoints alongside production:

{
  "providers": {
    "perplexity": {
      "apiKey": "pplx-...",
      "baseUrl": "https://staging.api.perplexity.ai/search"
    }
  }
}

6. Switching Perplexity between pure search and Sonar

Perplexity has two endpoints:

You can override either with a custom baseUrl:

{
  "providers": {
    "perplexity": {
      "apiKey": "pplx-...",
      "baseUrl": "https://api-proxy.example.com/perplexity/search"
    }
  }
}

When model is set, the override is interpreted as the chat-completions URL; otherwise the pure search URL.

7. Self-hosted SearXNG as a stand-in for any search provider

If you operate a SearXNG instance that exposes a JSON API and you want the skill to use it as a "Tavily-shaped" backend, you'd:

1. Run a small adapter that translates Tavily's request body to SearXNG's ?format=json query and reshapes the response. 2. Point Tavily's baseUrl at the adapter:

{
  "providers": {
    "tavily": {
      "apiKey": "ignored-by-adapter",
      "baseUrl": "https://searxng-adapter.example.internal/search"
    }
  }
}

The adapter does the translation; the skill remains unchanged.

Per-provider notes

Verification

After setting an override, verify with --list-providers:

python3 scripts/web-search --list-providers

The output marks overridden providers with (baseUrl-overridden):

[OK] tavily        https://eu.api.tavily.com/search  (baseUrl-overridden)

Then run a test query:

python3 scripts/web-search --provider tavily "kubernetes deployment" --json | jq '.manifests[0].endpoint'

The endpoint field in the manifest reflects the actual URL called — if the override didn't take effect, it shows the default URL.

Failure modes

The skill does not validate the override URL. If you point at the wrong path:

• HTTP-level errors (404, 405, 410) surface in the manifest's errors[] with the upstream status code.
• Schema mismatches surface as Failed to parse response: ... in the same errors[] list.

Both cases preserve the raw response in <provider>-raw.json so you can inspect what came back. Use --check to confirm credentials are present, but the only way to validate the override URL is to run a real query.

Reverting

Remove the baseUrl field from the provider's config block, or set it to null / empty string. The skill falls back to the upstream default for that provider on the next call.

curl Recipes — Python-less Fallback

Every provider can be invoked directly with curl if Python is not available (some Windows machines, distroless containers, jailed CI runners). The recipes below are the exact shape the skill itself sends. Copy, paste, replace the placeholder, and pipe through jq (or python3 -c "import json" if you have neither jq nor a wrapper).

The recipes assume bash-style variable substitution. On PowerShell, replace ${VAR} with $env:VAR. On cmd.exe, replace with %VAR%. The HTTP body is identical.

Saving the response to a temp file

TMPDIR=/tmp/web-search-manual
mkdir -p "$TMPDIR"
RESULT="$TMPDIR/$(date -u +%Y%m%dT%H%M%SZ).json"
# (any curl below) ... -o "$RESULT"
jq '.' "$RESULT"

Or pipe directly through jq/rg:

curl ... | jq '.results[].url'
curl ... | rg -i "kubernetes"

---

duckduckgo (free, no auth)

curl -fsSL --get \
  --data-urlencode "q=python programming language" \
  --data "format=json&no_html=1&skip_disambig=1&no_redirect=1" \
  "https://api.duckduckgo.com/" \
  | jq '.RelatedTopics[] | {title: .Text, url: .FirstURL}'

mwmbl (free, no auth)

curl -fsSL --get \
  --data-urlencode "s=rust programming" \
  "https://api.mwmbl.org/api/v1/search/" \
  | jq '.[] | {title: ([.title[].value] | join("")), url: .url}'

exa

curl -fsSL -X POST "https://api.exa.ai/search" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${EXA_API_KEY}" \
  -d '{"query": "kubernetes 1.31 release notes", "numResults": 10}' \
  | jq '.results[] | {title, url, score}'

Anonymous calls are no longer accepted (return 402 Payment Required since 2026).

tavily

curl -fsSL -X POST "https://api.tavily.com/search" \
  -H "Authorization: Bearer ${TAVILY_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"query": "rust async runtime comparison", "max_results": 10}' \
  | jq '.results[] | {title, url, content}'

brave

curl -fsSL --get \
  --data-urlencode "q=postgresql vs mysql 2026 site:stackoverflow.com" \
  --data "count=10" \
  -H "Accept: application/json" \
  -H "X-Subscription-Token: ${BRAVE_API_KEY}" \
  "https://api.search.brave.com/res/v1/web/search" \
  | jq '.web.results[] | {title, url, snippet: .description}'

serper

curl -fsSL -X POST "https://google.serper.dev/search" \
  -H "X-API-KEY: ${SERPER_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"q": "elasticsearch alternatives", "num": 10}' \
  | jq '.organic[] | {title, url: .link, snippet}'

google-cse

curl -fsSL --get \
  --data-urlencode "q=jq tutorial" \
  --data "key=${GOOGLE_CSE_API_KEY}&cx=${GOOGLE_CSE_ID}&num=10" \
  "https://customsearch.googleapis.com/customsearch/v1" \
  | jq '.items[] | {title, url: .link, snippet}'

Max 10 results per call. Hard limit on the free tier of 100 queries/day.

z-ai

curl -fsSL -X POST "https://api.z.ai/api/paas/v4/web_search" \
  -H "Authorization: Bearer ${Z_AI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"search_engine": "search-prime", "search_query": "ChatGLM open source", "count": 10}' \
  | jq '.search_result[] | {title, url: .link, snippet: .content}'

perplexity — pure search (no model)

curl -fsSL -X POST "https://api.perplexity.ai/search" \
  -H "Authorization: Bearer ${PERPLEXITY_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "open source vector database 2026",
    "max_results": 10,
    "search_recency_filter": "month"
  }' \
  | jq '.results[] | {title, url, snippet, date}'

perplexity — sonar (chat completions)

curl -fsSL -X POST "https://api.perplexity.ai/chat/completions" \
  -H "Authorization: Bearer ${PERPLEXITY_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "sonar-pro",
    "messages": [{"role": "user", "content": "Compare Pinecone vs Weaviate vs Qdrant"}]
  }' \
  | jq '{answer: .choices[0].message.content, sources: .citations}'

xai (Grok web_search tool)

curl -fsSL -X POST "https://api.x.ai/v1/responses" \
  -H "Authorization: Bearer ${XAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "grok-4.3",
    "input": "latest news about EU AI Act",
    "tools": [{"type": "web_search"}],
    "tool_choice": "required"
  }' \
  | jq '.output[] | select(.type=="message") | .content[].annotations[]? | {title, url}'

To suppress inline citation markers in the answer prose, add "include": ["no_inline_citations"] to the body.

openai (Responses API + web_search tool)

curl -fsSL -X POST "https://api.openai.com/v1/responses" \
  -H "Authorization: Bearer ${OPENAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.4-mini",
    "input": "what was a positive news story from today",
    "tools": [{
      "type": "web_search",
      "search_context_size": "medium",
      "external_web_access": true
    }],
    "tool_choice": "required",
    "max_tool_calls": 3
  }' \
  | jq '.output[] | select(.type=="message") | .content[].annotations[]? | {title, url}'

anthropic (Messages API + web_search tool)

curl -fsSL -X POST "https://api.anthropic.com/v1/messages" \
  -H "x-api-key: ${ANTHROPIC_API_KEY}" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-4-6",
    "max_tokens": 2048,
    "messages": [{"role": "user", "content": "Latest research on transformer interpretability"}],
    "tools": [{
      "type": "web_search_20260209",
      "name": "web_search",
      "max_uses": 3
    }]
  }' \
  | jq '.content[] | select(.type=="web_search_tool_result") | .content[] | {title, url, page_age}'

---

Multi-provider parallel without the wrapper

If you have bash but not Python, run providers concurrently with &:

QUERY="elasticsearch alternatives"
TMPDIR=/tmp/web-search-manual/$(date -u +%Y%m%dT%H%M%SZ)
mkdir -p "$TMPDIR"

(curl -fsSL -X POST https://api.tavily.com/search \
  -H "Authorization: Bearer $TAVILY_API_KEY" -H "Content-Type: application/json" \
  -d "{\"query\":\"$QUERY\",\"max_results\":10}" \
  | jq '{provider:"tavily", results:[.results[] | {title,url,snippet:.content}]}' \
  > "$TMPDIR/tavily.json") &

(curl -fsSL --get --data-urlencode "q=$QUERY" --data "count=10" \
  -H "X-Subscription-Token: $BRAVE_API_KEY" \
  https://api.search.brave.com/res/v1/web/search \
  | jq '{provider:"brave", results:[.web.results[] | {title,url,snippet:.description}]}' \
  > "$TMPDIR/brave.json") &

(curl -fsSL --get --data-urlencode "q=$QUERY" --data "format=json&no_html=1" \
  https://api.duckduckgo.com/ \
  | jq '{provider:"duckduckgo", results:[.RelatedTopics[]? | {title:.Text, url:.FirstURL, snippet:.Text}]}' \
  > "$TMPDIR/duckduckgo.json") &

wait

# Combine all into one ranked, deduped list:
jq -s '
  {query: "'"$QUERY"'", manifests: ., results: [.[].results[]] | unique_by(.url)}
' "$TMPDIR"/*.json > "$TMPDIR/combined.json"

echo "Done: $TMPDIR/combined.json"
jq '.results[] | .url' "$TMPDIR/combined.json"

This is exactly what the Python wrapper does internally. Use whichever path is more convenient for your environment.

---

When neither curl nor python3 is available

This should be vanishingly rare on modern Unix systems and Windows 10+, but if you're stuck:

The HTTP semantics are identical to the curl recipes above.

Multi-Provider Strategies

The skill supports three execution modes: single provider, named subset, or every-configured-provider fan-out. Each mode runs requests in a thread pool — multi-provider calls are concurrent, not serial.

Modes

--provider, --providers, --all all override the config's fallback chain. The chain only kicks in when none of those flags are passed.

How parallelism works

The skill uses ThreadPoolExecutor from the stdlib. Every provider call is one HTTP request, so threads (not processes) are correct — they share network bandwidth and complete asynchronously while waiting for I/O.

--parallelism N        Max concurrent provider calls (default 8).

For --all with 8 providers configured, all 8 fire in parallel. With 12 configured and --parallelism 4, the runner queues four at a time. The pool shuts down cleanly on Ctrl-C or after all results return.

Each provider writes its own <provider>.json and <provider>-raw.json independently, so partial failures do not break the run.

Why fan out?

Combining results

Each provider's results live independently. To merge them client-side:

RESULT=$(python3 scripts/web-search --all --print-path-only "...")

# Dedupe by URL, take top 10 by score where available:
jq '
  [.manifests[].results[]]
  | unique_by(.url)
  | sort_by(-(.score // 0))
  | .[:10]
' "$RESULT"

For a "consensus rank" where URLs hit by multiple providers float to the top:

jq '
  [.manifests[].results[].url]
  | group_by(.)
  | map({url: .[0], count: length})
  | sort_by(-.count)
' "$RESULT"

Pair the consensus URLs with their best-quality snippets:

jq '
  [.manifests[].results[]]
  | group_by(.url)
  | map({
      url: .[0].url,
      hits: length,
      providers: [.[] | (.source // "?")] | unique,
      title: .[0].title,
      snippet: ([.[] | .snippet // ""] | max_by(length))
    })
  | sort_by(-.hits)
' "$RESULT"

Pre-built strategies

"Free first, paid second"

# Free providers only — instant, zero-cost:
python3 scripts/web-search --providers duckduckgo,mwmbl "..."

# If results were thin, escalate to a paid provider:
python3 scripts/web-search --providers tavily,brave "..."

You can wrap this in a tiny shell function:

ws() {
  local query="$1"
  local result
  result=$(python3 scripts/web-search --providers duckduckgo,mwmbl --print-path-only "$query")
  local count
  count=$(jq '[.manifests[].results[]] | length' "$result")
  if [ "$count" -lt 5 ]; then
    echo "Free providers thin ($count results) — escalating to paid"
    result=$(python3 scripts/web-search --providers tavily,brave --print-path-only "$query")
  fi
  jq '.manifests[].results[] | {title, url}' "$result"
}

"Independent + proxied Google"

python3 scripts/web-search --providers brave,serper "..."

Brave's index is independent; Serper proxies Google. Together you cover the two main kinds of web index in 2026.

"AI-mediated triangulation"

python3 scripts/web-search --providers tavily,perplexity,anthropic "..."

Tavily gives you raw cleaned snippets. Perplexity (in pure mode) gives you a separate index. Anthropic gives you Claude-curated, reasoning-filtered citations. Disagreements between the three are diagnostic — they often surface different angles on a question.

"Cost-bounded sweep"

python3 scripts/web-search --all --max-results 3 "..."

--all fans out, but the per-provider --max-results 3 keeps token cost low for the model-mediated providers (xAI, OpenAI, Anthropic). You still get 3 × N URLs to merge.

Sequential fallback chain (config-driven)

Add fallback to your config and the skill runs providers in order, stopping at the first that returns results:

{
  "default": "duckduckgo",
  "fallback": ["mwmbl", "brave", "tavily", "anthropic", "openai"]
}

Recommended ordering: native zero-config providers first (so the skill works without any key), then paid raw-search, then LLM-backed.

Behavior:

1. Start with default. If it errors (HTTP non-2xx, network error, etc.) or returns zero results, fall through. 2. Walk fallback in order. Providers without configured credentials are silently skipped (logged as (skipped: no credentials) in --list-providers). 3. Stop at the first provider that produces a non-empty results[]. 4. Footer shows MODE: fallback and FALLBACK_TRIED: <names> listing every slot that fell through.

--list-providers displays the resolved chain at the bottom:

Fallback chain:   duckduckgo -> mwmbl -> brave -> tavily (skipped: no credentials) -> anthropic (skipped: no credentials) -> openai (skipped: no credentials)

When to use a fallback chain

Difference vs --all

A chain is the right choice for "I want one good answer cheaply"; --all is the right choice for "I want every angle". Pick based on what the caller is going to do with the result.

Fall-through triggers

The chain advances when the current provider:

• returns a non-2xx HTTP status
• fails to reach the network (DNS / TLS / timeout)
• returns 200 but with results: [] (empty result set)
• has no credentials in the config (silently skipped, no error logged)

The chain does not advance on:

• 200 with at least one result (success — return immediately)
• partial successes (e.g. 5 out of 10 results) — these still count as success

If you want "always run the chain to completion regardless of partial success", use --providers a,b,c instead and merge the results with jq.

Error isolation

When --all runs and provider X errors out (bad key, rate limit, network blip), the skill:

1. Records the error in combined.json under errors[]. 2. Still saves the partial result manifest under manifests[] for the providers that succeeded. 3. Exits with code 0 if at least one provider returned results, 1 if every provider failed.

You can spot errored providers without re-reading the whole file:

jq '.errors' "$RESULT"

The footer also lists them:

ERRORED: brave,xai

Concurrency tuning

The skill never spawns more threads than the number of providers in the call.

Caching across providers

Each provider's HTTP call is unique — there is no sensible way to cache one provider's response and reuse it for another. If you want to cache the combined result for a given query, hash the query string and store the manifest:

QUERY="kubernetes 1.31 release notes"
KEY=$(printf "%s" "$QUERY" | shasum | head -c 12)
CACHE="/tmp/web-search-cache/$KEY.json"
mkdir -p "$(dirname "$CACHE")"
[ -f "$CACHE" ] || python3 scripts/web-search --all --json "$QUERY" > "$CACHE"
jq '.manifests[].results[].url' "$CACHE"

When NOT to fan out

--all is wasteful for:

Pipeline Recipes

The skill is designed to be Unix-pipeable. Every search writes JSON files to a temp directory and prints a path to stdout, so you can chain rg, jq, grep, awk, python -c, or any other tool downstream.

Output anatomy

A single search creates a directory like /tmp/web-search/20260501T073744Z/ with these files:

/tmp/web-search/<run-id>/
├── combined.json           # Manifest with metadata + all normalized results across providers
├── <provider>.json         # Per-provider normalized results (canonical schema)
└── <provider>-raw.json     # Per-provider raw API response (debug + advanced parsing)

combined.json is the canonical entry point for downstream pipelines. <provider>.json is convenient for single-provider queries.

The schema is in `result-schema.md`.

Capturing the path

Three idiomatic ways:

# 1. --print-path-only emits ONLY the combined.json path (clean for $(...) substitution)
RESULT=$(python3 scripts/web-search --print-path-only "kubernetes 1.31 release notes")
jq '.manifests[].results[].url' "$RESULT"

# 2. The default human-readable output ends with a "--- WEBSEARCH ---" footer; grep for the path
python3 scripts/web-search "..." | awk '/^RESULT_FILE:/ {print $2}'

# 3. --json prints the full manifest to stdout — pipe directly into jq without disk
python3 scripts/web-search --json "..." | jq '.manifests[].results[].url'

ripgrep recipes

rg reads every file in the run directory by default — no need to know the provider name.

Filter results to lines containing a keyword

RUN=$(python3 scripts/web-search --providers tavily,brave --print-path-only "release notes" | xargs dirname)
rg -i "stable|release" "$RUN"

Find URLs from a specific domain

rg -o "https://[a-zA-Z0-9./_-]*github.com[a-zA-Z0-9./_-]*" "$RUN"

Show snippets containing a phrase, with surrounding result block

rg -B 2 -A 2 -i "vulnerability" "$RUN"/*.json

Pipe straight from --json

python3 scripts/web-search --json "..." | rg -i "deprecated"

jq recipes

combined.json (and the per-provider files) are valid JSON, so jq works directly.

Extract every URL across all providers

RESULT=$(python3 scripts/web-search --all --print-path-only "...")
jq -r '.manifests[].results[].url' "$RESULT"

Top result per provider

jq -r '.manifests[] | "\(.provider): \(.results[0].title) - \(.results[0].url)"' "$RESULT"

Aggregate across providers, deduped by URL, ranked by score where available

jq '
  [.manifests[].results[]]                # flatten everything
  | unique_by(.url)                       # dedupe by URL
  | sort_by(-(.score // 0))               # rank by score (missing => 0)
  | .[:10]
' "$RESULT"

Filter by provider

jq '.manifests[] | select(.provider == "tavily").results' "$RESULT"

Convert to TSV for spreadsheet pasting

jq -r '
  .manifests[].results[]
  | [.title, .url, (.snippet // "")] | @tsv
' "$RESULT"

Extract published dates, filter to last 30 days

jq --arg cutoff "$(date -u -v-30d +%Y-%m-%d 2>/dev/null || date -u -d '30 days ago' +%Y-%m-%d)" '
  [.manifests[].results[]
   | select(.publishedAt? and (.publishedAt[:10] >= $cutoff))]
' "$RESULT"

Count results per provider

jq '.manifests | map({provider, count: (.results | length)})' "$RESULT"

Find providers that errored

jq '.errors' "$RESULT"

Build a markdown digest

jq -r '
  .manifests[]
  | "## \(.provider)\n",
    (.results[] | "- [\(.title)](\(.url))\n  > \(.snippet // "(no snippet)")\n")
' "$RESULT"

Combining tools

Open every URL in your browser

jq -r '.manifests[].results[].url' "$RESULT" | xargs -I{} open "{}"   # macOS
jq -r '.manifests[].results[].url' "$RESULT" | xargs -I{} xdg-open "{}"  # Linux

Send the top 5 to an LLM for summarization

jq -r '
  .manifests[].results[:5][]
  | "TITLE: \(.title)\nURL: \(.url)\n\(.snippet // "")\n---"
' "$RESULT" | llm "Summarize these results in 3 bullets."

(Replace llm with whatever CLI your harness uses.)

Snapshot results for offline reading

jq -r '.manifests[].results[].url' "$RESULT" | xargs -I{} \
  curl -fsSL --max-time 10 "{}" -o "snapshot/$(echo "{}" | sha1sum | head -c 12).html"

Diff two searches

A=$(python3 scripts/web-search --print-path-only "rust async")
B=$(python3 scripts/web-search --print-path-only "rust tokio")
diff <(jq -r '.manifests[].results[].url' "$A" | sort) \
     <(jq -r '.manifests[].results[].url' "$B" | sort)

Streaming a JSON Lines feed

For tools that prefer NDJSON over a single JSON object:

jq -c '.manifests[].results[]' "$RESULT" > /tmp/results.ndjson

Now every line is a single result, suitable for parallel, xargs -L 1, or stream-processing jobs.

When you have neither rg nor jq

Pure POSIX fallback:

RESULT=$(python3 scripts/web-search --print-path-only "...")
python3 -c "import json,sys; [print(r['url']) for m in json.load(open(sys.argv[1]))['manifests'] for r in m['results']]" "$RESULT"

python3 -c is the most universal substitute — it ships with Python on every Unix and runs from PowerShell/cmd.exe on Windows.

Pretty-print without paging

The skill never invokes less or any pager. Pipe to cat if you want to ensure no terminal-aware behavior:

python3 scripts/web-search "..." | cat

Background searches

For long-running multi-provider fan-outs, run in the background and wait for the sentinel file:

RUN_DIR=/tmp/web-search/$(date -u +%Y%m%dT%H%M%SZ)-bg
nohup python3 scripts/web-search --all --tmp-dir "$RUN_DIR" "..." \
  > "$RUN_DIR.log" 2>&1 &
echo "Started $!"

# In another shell or after waiting:
tail -f "$RUN_DIR.log"
ls "$RUN_DIR"      # combined.json appears when finished

Caching across runs

The skill never caches between invocations — every call writes a fresh directory under /tmp/web-search/. To deduplicate identical queries, build your own cache layer:

QUERY="kubernetes 1.31 release notes"
KEY=$(printf "%s" "$QUERY" | sha1sum | head -c 12)
CACHE="/tmp/web-search-cache/$KEY.json"
mkdir -p "$(dirname "$CACHE")"
if [ ! -f "$CACHE" ]; then
  python3 scripts/web-search --json "$QUERY" > "$CACHE"
fi
jq '.manifests[].results[].url' "$CACHE"

/tmp/web-search/ itself is meant to be ephemeral — clear it whenever you want:

rm -rf /tmp/web-search/    # safe; the skill recreates the directory on next run

Platform Notes

The skill is designed to run on every modern OS without modification. This page documents the small differences between platforms and the failure modes you might hit on each.

What the skill needs

That's it. No pip install, no node, no jq, no curl required to run the wrapper. (jq and rg are recommended for downstream filtering — see `pipelines.md`.)

Python availability

Invocation per OS

macOS / Linux

The shebang #!/usr/bin/env python3 makes the file directly executable:

chmod +x scripts/web-search
./scripts/web-search "query"
# or:
python3 scripts/web-search "query"

Windows (cmd.exe)

The shebang is ignored; invoke Python explicitly:

python scripts\web-search "query"
:: or, if the py launcher is available:
py scripts\web-search "query"

Windows (PowerShell)

Same as cmd, plus PowerShell-friendly variable substitution if you set env vars:

$env:TAVILY_API_KEY = "tvly-..."
python scripts\web-search "your query"

Windows (Git Bash / WSL)

Behaves like Linux; the shebang works:

./scripts/web-search "query"

Path resolution per OS

The skill resolves three classes of paths automatically.

Config file search order

Confirm with --list-providers, which prints the resolved source.

Temp output directory

The temp dir is created lazily on first run and never auto-cleaned. Wipe it with rm -rf (POSIX) or Remove-Item -Recurse (PowerShell) when you want to free space.

Run-directory paths in output

combined.json and the footer print absolute paths using the host's native separator:

• POSIX: /tmp/web-search/20260501T073745Z/combined.json
• Windows: C:\Users\foo\AppData\Local\Temp\web-search\20260501T073745Z\combined.json

jq and rg accept both. If you script around the output, treat the path as opaque (do not split on /).

SSL and certificates

Python's urllib.request uses the system trust store on each OS:

• macOS: Uses the keychain via Security.framework (since Python 3.6+).
• Linux: Reads from /etc/ssl/certs/ or distro-specific bundle.
• Windows: Uses the OS certificate store via wincertstore (Python 3.4+).

Corporate proxies that re-sign HTTPS need their root CA installed in the OS store. The skill respects that automatically — no special config needed.

If you hit a TLS error in a sandboxed environment:

# Diagnose: which CA bundle is being used?
python3 -c "import ssl; print(ssl.get_default_verify_paths())"

If openafile or cafile is empty/missing, the OS trust store is broken — fix that first (e.g. pip install certifi in a venv as a fallback, then export SSL_CERT_FILE=$(python3 -m certifi)).

Locale and encoding

The skill writes JSON in UTF-8 and reads stdin/stdout in UTF-8. On Windows, ensure your terminal is set to UTF-8 to avoid mojibake on non-ASCII results:

chcp 65001       # set the cmd.exe code page to UTF-8 for the session

Modern Windows Terminal / VS Code Terminal are UTF-8 by default. The legacy cmd.exe opened from Win+R cmd may still default to cp1252 — switch to Windows Terminal if you see garbled snippets from Chinese-language results (z.ai), Korean queries, etc.

Threading and process model

ThreadPoolExecutor works identically on all OSes. The skill never forks subprocesses, so:

• No fork() / spawn() differences between macOS and Linux.
• No issues with macOS's OBJC_DISABLE_INITIALIZE_FORK_SAFETY warning.
• No Windows multiprocessing pickling quirks.

This is intentional — every provider call is one urllib request, and threads are the right primitive.

Shell shebang on Windows

The shebang #!/usr/bin/env python3 is a comment from Windows's perspective. The file is plain text, and Windows file association decides what runs it:

• File Explorer double-click: opens .py files with the default Python launcher if one is installed (we don't ship a .py extension, so this won't auto-resolve).
• python scripts\web-search "...": explicit, always works.
• py scripts\web-search "...": works if the py launcher (Python.org installers) is available.

If you want to wire up web-search "..." as a top-level command on Windows, drop a small batch file web-search.cmd in your PATH:

@echo off
python "C:\path\to\skill\scripts\web-search" %*

Or a PowerShell function in your $PROFILE:

function web-search { python "C:\path\to\skill\scripts\web-search" @Args }

Containerization

Minimal Dockerfile that includes the skill plus jq and rg for the full pipeline experience:

FROM python:3.12-slim
RUN apt-get update && apt-get install -y --no-install-recommends jq ripgrep ca-certificates && rm -rf /var/lib/apt/lists/*
COPY . /skill
WORKDIR /skill
RUN chmod +x scripts/web-search
ENTRYPOINT ["python3", "scripts/web-search"]

docker build -t web-search .
docker run --rm \
  -e TAVILY_API_KEY="$TAVILY_API_KEY" \
  -v /tmp/web-search:/tmp/web-search \
  web-search "kubernetes 1.31 release notes"

Mounting /tmp/web-search lets the host script consume the result files written inside the container. Pass keys via -e ENV_VAR or mount a config:

docker run --rm \
  -v $HOME/.config/web-search:/root/.config/web-search:ro \
  -v /tmp/web-search:/tmp/web-search \
  web-search "..."

Known platform-specific gotchas

Verifying portability

Quick self-test that works on every supported OS:

python3 scripts/web-search --version
python3 scripts/web-search --list-providers
python3 scripts/web-search --check
python3 scripts/web-search --provider duckduckgo "hello world"

If the last line completes and writes a result file, the skill is ready on this host. The first three commands produce no network traffic — they only validate the config and print metadata.

anthropic

Anthropic Claude web search via the Messages API + web_search server tool. Model-mediated only — there is no raw-search alternative.

At a glance

Important: no raw search alternative

Anthropic does not offer any raw search endpoint or /v1/search route. The Messages API + web_search tool is the only path. Confirmed against:

• The complete /v1/* endpoint list at docs.anthropic.com/en/api/overview
• The official TypeScript and Python SDKs (no search() method on the client)
• The 2025 "Web Search API" announcement (which described the tool addition, not a new endpoint)

If you need model-less search, use tavily, brave, serper, or perplexity instead.

When to use

Setup

1. Get an API key from console.anthropic.com/settings/keys (sk-ant-…). 2. Make sure your account has access to a model that supports the web_search tool (Claude Opus 4.x, Sonnet 4.x, Haiku 4.x). 3. Config:

   {
     "providers": {
       "anthropic": {
         "apiKey": "sk-ant-...",
         "model": "claude-sonnet-4-6",
         "maxUses": 5
       }
     }
   }

4. Env: ANTHROPIC_API_KEY.

Optional config fields:

• model (default claude-sonnet-4-6) — see supported list below
• toolVersion (default web_search_20260209) — pin to web_search_20250305 for ZDR
• maxUses (default 5) — cap how many search calls Claude can make per request
• maxTokens (default 2048) — max_tokens in the request body
• userLocation — {country, region, city, timezone}

Tool versions

Supported models

Older Claude 3.x models support only web_search_20250305.

Request

POST https://api.anthropic.com/v1/messages
x-api-key: <key>
anthropic-version: 2023-06-01
Content-Type: application/json

{
  "model": "claude-sonnet-4-6",
  "max_tokens": 2048,
  "messages": [{"role": "user", "content": "<query>"}],
  "tools": [{
    "type": "web_search_20260209",
    "name": "web_search",
    "max_uses": 5,
    "allowed_domains": ["bloomberg.com", "sec.gov"],   ; OR `blocked_domains` (mutex)
    "user_location": {
      "type": "approximate",
      "country": "US",
      "region": "California",
      "city": "San Francisco",
      "timezone": "America/Los_Angeles"
    }
  }]
}

Response

{
  "id": "msg_...",
  "content": [
    {"type": "text", "text": "I'll search for..."},
    {
      "type": "server_tool_use",
      "id": "srvtoolu_...",
      "name": "web_search",
      "input": {"query": "..."}
    },
    {
      "type": "web_search_tool_result",
      "tool_use_id": "srvtoolu_...",
      "content": [{
        "type": "web_search_result",
        "url": "https://...",
        "title": "...",
        "encrypted_content": "...",
        "page_age": "April 30, 2025"
      }]
    },
    {
      "type": "text",
      "text": "Final answer with cited sources.",
      "citations": [{
        "type": "web_search_result_location",
        "url": "https://...",
        "title": "...",
        "encrypted_index": "...",
        "cited_text": "Up to 150 chars of cited content."
      }]
    }
  ],
  "usage": {"server_tool_use": {"web_search_requests": 1}}
}

Field mapping (canonical schema)

The skill walks content[] collecting: 1. Raw results from web_search_tool_result.content[] blocks → url, title, page_age 2. Citations from text.citations[] → enriches matching URLs with cited_text (up to 150 chars)

Domain filtering

Native structured fields — allowed_domains and blocked_domains. Mutually exclusive.

Wildcard support:

• No scheme: example.com, NOT https://example.com
• Subdomains auto-included: example.com covers docs.example.com
• Specific subdomain restricts: docs.example.com
• Subpaths: example.com/blog
• One * wildcard allowed in path: example.com/*

python3 scripts/web-search --provider anthropic --include docs.anthropic.com "..."

baseUrl override

{"providers": {"anthropic": {
  "apiKey": "sk-ant-...",
  "model": "claude-sonnet-4-6",
  "baseUrl": "https://anthropic-gw.bedrock.us-east-1.amazonaws.com/v1/messages"
}}}

Common patterns:

• AWS Bedrock-hosted Anthropic: https://anthropic-gw.bedrock.<region>.amazonaws.com/v1/messages
• Azure-hosted: https://anthropic-gw.azure.example.com/v1/messages
• Vertex AI proxy: https://anthropic-vertex.example.com/v1/messages
• Corporate LLM gateway: https://llm-gw.internal/anthropic/v1/messages

The override must accept the standard Messages API body shape. See `../base-urls.md`.

curl recipe (Python-less fallback)

curl -fsSL -X POST "https://api.anthropic.com/v1/messages" \
  -H "x-api-key: ${ANTHROPIC_API_KEY}" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-4-6",
    "max_tokens": 2048,
    "messages": [{"role": "user", "content": "Latest research on transformer interpretability"}],
    "tools": [{
      "type": "web_search_20260209",
      "name": "web_search",
      "max_uses": 3
    }]
  }' \
  | jq '.content[] | select(.type=="web_search_tool_result") | .content[] | {title, url, page_age}'

Quirks

• Snippets are tricky. web_search_result.encrypted_content is opaque — the readable text only appears in text.citations[].cited_text (max 150 chars per citation). The skill matches citations to results by URL.
• `page_age` is a human-readable string ("April 30, 2025") — not an ISO date. The skill copies it verbatim to publishedAt. Parse downstream if you need a real date.
• Server tool_use IDs use srvtoolu_ prefix (vs client-side toolu_).
• `pause_turn` stop reason can occur for complex multi-search queries — you must send a continuation request. The skill does NOT handle this automatically (it surfaces the error).
• Multi-turn citations: pass back encrypted_content and encrypted_index from previous results to maintain citations across turns. The skill is single-turn so this does not apply.
• No score field. Anthropic does not provide relevance scores.
• Dynamic filtering (web_search_20260209) requires the code execution tool internally — it is not ZDR-eligible by default. Pin toolVersion: web_search_20250305 if you need ZDR.

Pricing

$10 per 1,000 searches plus standard model token costs. Search results count as input tokens in current and subsequent turns.

Cost-control tips:

• Set maxUses: 3 (or lower) to cap agentic loops
• Use claude-haiku-4-5 for cheaper synthesis (no _20260209 support)
• Pin toolVersion: web_search_20250305 if dynamic filtering's ~24% token reduction does not justify ZDR loss

See docs.anthropic.com/en/docs/about-claude/pricing.

See also

• Pipelines, Multi-provider, Result schema, baseUrl
• For raw-search alternatives: perplexity.md, tavily.md, brave.md, serper.md

brave

Brave Search API — independent web index. Privacy-focused, no Google/Bing dependency.

At a glance

When to use

Setup

1. Go to api-dashboard.search.brave.com/app/subscriptions and create an account. 2. Subscribe to the Free tier ($5/month credit). Credit card required as of Feb 2026. 3. Create an API key from the dashboard. Token is sometimes prefixed BSA…. 4. Config:

   {"providers": {"brave": {"apiKey": "BSA..."}}}

Or env: BRAVE_API_KEY (also BRAVE_SEARCH_API_KEY).

Free-tier rate limit: 1 req/sec. Paid plans go up to 50 req/sec.

Request

GET https://api.search.brave.com/res/v1/web/search?q=<query>&count=<n>
Accept: application/json
X-Subscription-Token: <key>

The skill builds the query string with parse_qsl so any pre-existing query parameters on a baseUrl override are preserved.

Response

{
  "web": {
    "results": [
      {
        "title": "...",
        "url": "https://...",
        "description": "...",
        "page_age": "2025-08-01T12:00:00"
      }
    ]
  },
  "query": {...},
  "infobox": {...}
}

Field mapping (canonical schema)

Domain filtering

Brave does not accept structured domain filter fields. The skill appends site: and -site: operators to the query.

python3 scripts/web-search --provider brave --include docs.brave.com "..."

baseUrl override

{"providers": {"brave": {"apiKey": "BSA...", "baseUrl": "https://search-proxy.example.internal/brave"}}}

The override URL must accept q= and count= query params. See `../base-urls.md`.

curl recipe (Python-less fallback)

curl -fsSL --get \
  --data-urlencode "q=postgresql vs mysql 2026" \
  --data "count=10" \
  -H "Accept: application/json" \
  -H "X-Subscription-Token: ${BRAVE_API_KEY}" \
  "https://api.search.brave.com/res/v1/web/search" \
  | jq '.web.results[] | {title, url, snippet: .description}'

Quirks

• Credit-card-required free tier since Feb 2026. Truly free use is not possible without billing setup.
• Returns rich infoboxes and "discussions" sections. The skill ignores these and uses only web.results[].
• page_age is an ISO timestamp string — usable directly in publishedAt filters.

Pricing

$5 / 1,000 queries on the standard tier. Higher tiers for production volume — see brave.com/search/api.

See also

• Pipelines, Multi-provider, Result schema

duckduckgo

DuckDuckGo Instant Answer API. Free, no API key, official public endpoint. Best for factual lookups ("what is X", definitions, calculations, named-entity disambiguation).

At a glance

When to use

Setup

None. The skill calls the Instant Answer API anonymously.

Request

The skill issues a GET to https://api.duckduckgo.com/ with the following query parameters:

Headers: Accept: application/json, User-Agent: web-search-skill/<version>.

Response

{
  "Heading": "Python (programming language)",
  "AbstractText": "Python is a high-level, general-purpose programming language...",
  "AbstractURL": "https://en.wikipedia.org/wiki/Python_(programming_language)",
  "AbstractSource": "Wikipedia",
  "Answer": "",
  "AnswerType": "",
  "Definition": "",
  "DefinitionURL": "",
  "RelatedTopics": [
    {"Text": "NumPy - NumPy is a library for...", "FirstURL": "https://duckduckgo.com/NumPy"},
    {"Topics": [/* nested category groupings */]}
  ],
  "Results": [{"FirstURL": "...", "Text": "..."}]
}

Field mapping (canonical schema)

The normalizer recursively walks RelatedTopics[].Topics[] to flatten category groupings.

Domain filtering

DuckDuckGo Instant Answer does not accept structured domain filters. The skill appends site:example.com and -site:example.com operators to the q parameter, which the underlying search engine honors.

python3 scripts/web-search --provider duckduckgo --include en.wikipedia.org "python"

baseUrl override

{"providers": {"duckduckgo": {"baseUrl": "https://ddg-mirror.example.internal/"}}}

The override must accept the same query-string parameters listed above. The skill appends q=, format=json, etc. — your mirror should respect them. See `../base-urls.md`.

curl recipe (Python-less fallback)

curl -fsSL --get \
  --data-urlencode "q=python programming language" \
  --data "format=json&no_html=1&skip_disambig=1&no_redirect=1" \
  "https://api.duckduckgo.com/" \
  | jq '{heading: .Heading, abstract_url: .AbstractURL, related: [.RelatedTopics[]? | {title: .Text, url: .FirstURL}]}'

Quirks

• Not a ranked-links search. DuckDuckGo "Instant Answer" returns abstracts and topic summaries, not Google-style organic results. For full link sets pair with another provider via --providers duckduckgo,mwmbl,....
• For empty/unknown queries it returns a 200 with all fields blank — the skill yields an empty results[], not an error.
• RelatedTopics can nest category groupings under a Topics key. The normalizer flattens these.
• No published rate limits. The project asks consumers to use a meaningful User-Agent and stay "reasonable."

Pricing

Free. No quota. No credit card.

See also

• Pipelines (rg / jq recipes)
• Multi-provider strategies — pair with mwmbl for actual web links
• Result schema

exa

Exa — semantic / neural search. API key required as of 2026 (anonymous calls now return 402 Payment Required).

At a glance

When to use

Setup

1. Sign up at dashboard.exa.ai. 2. Generate an API key from the dashboard's API Keys page (exa_… format). 3. Drop into config or set the env var:

   {"providers": {"exa": {"apiKey": "exa_..."}}}

   export EXA_API_KEY="exa_..."

Note: Earlier versions of this skill claimed Exa accepted anonymous calls. Exa changed that policy in 2026 — calls without a key now return 402 Payment Required. The skill no longer treats Exa as zero-config.

Request

POST https://api.exa.ai/search
Content-Type: application/json
x-api-key: <key>

{
  "query": "<query string>",
  "numResults": 10,
  "includeDomains": ["..."],     ; optional
  "excludeDomains": ["..."]      ; optional
}

The skill clamps numResults to the server cap of 20.

Response

{
  "results": [
    {
      "title": "...",
      "url": "https://...",
      "text": "Full or summarized page content...",
      "score": 0.87,
      "publishedDate": "2025-09-15"
    }
  ],
  "autopromptString": "..."
}

Field mapping (canonical schema)

Domain filtering

Native structured fields. The skill passes --include to includeDomains and --exclude to excludeDomains.

python3 scripts/web-search --provider exa --include arxiv.org "transformer interpretability"

baseUrl override

{"providers": {"exa": {"apiKey": "exa_...", "baseUrl": "https://exa-proxy.internal/search"}}}

The override must accept the same JSON body. See `../base-urls.md`.

curl recipe (Python-less fallback)

# Anonymous (lower quota):
curl -fsSL -X POST "https://api.exa.ai/search" \
  -H "Content-Type: application/json" \
  -d '{"query": "kubernetes 1.31 release notes", "numResults": 10}'

# With key:
curl -fsSL -X POST "https://api.exa.ai/search" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${EXA_API_KEY}" \
  -d '{"query": "kubernetes 1.31 release notes", "numResults": 10}' \
  | jq '.results[] | {title, url, score}'

Quirks

• `text` field doubles as the snippet. It is sometimes long-form summaries — useful for downstream LLMs but heavier than typical search snippets.
• Anonymous calls return `402 Payment Required` as of 2026. The skill now treats Exa like any other paid provider.
• The optional autopromptString in the response is Exa's interpreted query — useful for debugging unexpected results.

Pricing

See the latest at exa.ai/pricing. Pay-as-you-go with no anonymous tier.

See also

• Pipelines (rg / jq recipes)
• Result schema
• baseUrl override

google-cse

Google Programmable Search Engine via the Custom Search JSON API. The official Google route — requires both an API key AND a custom search engine ID.

At a glance

When to use

Setup

This requires two values: an API key and a Programmable Search Engine ID.

1. Create a Programmable Search Engine at programmablesearchengine.google.com/controlpanel/all:

• Click Add
• Pick Search the entire web or specify domains
• Save and copy the Search engine ID (cx) from the Overview tab.

2. Get an API key:

• console.cloud.google.com/apis/credentials
• Create a project if needed
• + Create Credentials → API Key
• Restrict to Custom Search API if possible

3. Enable the API: console.cloud.google.com/apis/library/customsearch.googleapis.com → Enable 4. Config:

   {
     "providers": {
       "google-cse": {
         "apiKey": "AIza...",
         "searchEngineId": "017576662512468239146:omuauf_lfve"
       }
     }
   }

Env vars: GOOGLE_CSE_API_KEY (or GOOGLE_API_KEY) and GOOGLE_CSE_ID (or GOOGLE_SEARCH_ENGINE_ID).

Request

GET https://customsearch.googleapis.com/customsearch/v1
    ?q=<query>
    &key=<api-key>
    &cx=<search-engine-id>
    &num=10
Accept: application/json

The skill clamps num to 10 (hard server cap).

Response

{
  "items": [
    {
      "title": "...",
      "link": "https://...",
      "snippet": "...",
      "displayLink": "example.com"
    }
  ],
  "queries": {...},
  "searchInformation": {...}
}

Field mapping (canonical schema)

Domain filtering

Google CSE supports domain restriction at the engine level (configure in the PSE control panel). At call time, the skill appends site: / -site: operators to q for ad-hoc filtering.

python3 scripts/web-search --provider google-cse --include developers.google.com "..."

baseUrl override

{"providers": {"google-cse": {
  "apiKey": "AIza...",
  "searchEngineId": "...",
  "baseUrl": "https://gcse-proxy.internal/customsearch/v1"
}}}

The override must accept q=, key=, cx=, num= query params. See `../base-urls.md`.

curl recipe (Python-less fallback)

curl -fsSL --get \
  --data-urlencode "q=jq tutorial" \
  --data "key=${GOOGLE_CSE_API_KEY}&cx=${GOOGLE_CSE_ID}&num=10" \
  "https://customsearch.googleapis.com/customsearch/v1" \
  | jq '.items[] | {title, url: .link, snippet}'

Quirks

• Hard cap of 10 results per call — this is a Google policy, not the skill's choice. Pagination via start=11, start=21, etc. is possible but not implemented in the skill.
• API key sent as query parameter — be careful with logging. Use a CI secret manager.
• 100 queries/day free is the strictest free quota in this catalog. Plan around it.

Pricing

$5 / 1,000 queries beyond the 100/day free tier. Hard daily cap of 10,000 queries by default (raise via Google Cloud quota requests).

See also

• Pipelines, Multi-provider, Result schema

Providers — Index

The skill ships with 12 search providers behind a single CLI. This page is the routing layer: comparison table, selection guide, and pointers into the per-provider deep-dives. Each <provider>.md file in this directory is the authoritative source for that provider — endpoint, auth, request/response shape, setup steps, curl recipe, baseUrl override pattern, and quirks all in one place.

Quick comparison

Pure search? = "does the endpoint return raw ranked links without an LLM in the loop?". For model-mediated providers (xai/openai/anthropic) the LLM decides when, how often, and how to search — you trade direct control for synthesis and reasoning.

No raw search API exists for Anthropic or OpenAI as of May 2026. Both vendors gate web search behind their model-inference endpoints (Messages API / Responses API + web_search tool). Confirmed against current docs and SDKs. If you need model-less search results, use the dedicated providers above (Tavily, Brave, Serper, Exa, Perplexity, etc.).

How to choose

For research-style queries where you want diverse coverage, fan out across several:

python3 scripts/web-search --providers tavily,brave,perplexity "..."
python3 scripts/web-search --all "..."

See `../multi-provider.md` for parallel fan-out strategies and consensus ranking recipes.

baseUrl override (every provider)

Every provider supports a baseUrl config field that fully replaces the default endpoint. Use it for self-hosted proxies, regional mirrors, corporate gateways, or local mocks. Per-provider examples are inside each <provider>.md. The cross-cutting concept and patterns are documented in `../base-urls.md`.

Adding a new provider

The skill is intentionally easy to extend. To add a new provider in your fork:

1. Edit `scripts/web-search`:

• Add the name to PROVIDERS, DEFAULT_ENDPOINTS, and MAX_RESULTS_CAPS.
• Register an ENV_KEY_MAP entry for env-var fallback (or add to _NO_AUTH_PROVIDERS for no-auth).
• Implement build_<name>(cfg, query, max_results, allowed, blocked) returning (method, url, headers, body).
• Implement normalize_<name>(payload) returning a list of canonical result dicts.
• Register both in PROVIDER_BUILDERS and PROVIDER_NORMALIZERS.

2. Add a docs file at references/providers/<name>.md following the template — see any existing provider page. 3. Update this `index.md` comparison table.

The canonical result schema is documented in `../result-schema.md`.

mwmbl

MWMBL — community-crawled open-source independent search index. Free, no API key. Returns actual ranked web links (unlike duckduckgo's Instant Answers).

At a glance

When to use

Setup

None. The skill calls the public API anonymously.

Request

GET https://api.mwmbl.org/api/v1/search/?s=<query>

Headers: Accept: application/json, User-Agent: web-search-skill/<version>.

Response

The endpoint returns a top-level JSON array (the skill wraps it as {"_array": [...]} for uniform downstream handling).

[
  {
    "url": "https://en.wikipedia.org/wiki/Test",
    "title": [
      {"value": "Test", "is_bold": true},
      {"value": " - Wikipedia", "is_bold": false}
    ],
    "extract": [
      {"value": "Look up ", "is_bold": false},
      {"value": "test", "is_bold": true},
      {"value": " in Wiktionary, the free dictionary.", "is_bold": false}
    ],
    "source": "wikipedia"
  }
]

Field mapping (canonical schema)

Domain filtering

MWMBL has no structured domain filter. The skill appends site: and -site: operators to the query, which the upstream sometimes honors (best-effort).

python3 scripts/web-search --provider mwmbl --include rust-lang.org "async runtime"

baseUrl override

{"providers": {"mwmbl": {"baseUrl": "https://mwmbl-self-hosted.internal/api/v1/search/"}}}

If you self-host MWMBL the override points there directly. See `../base-urls.md`.

curl recipe (Python-less fallback)

curl -fsSL --get \
  --data-urlencode "s=rust programming" \
  "https://api.mwmbl.org/api/v1/search/" \
  | jq '.[] | {title: ([.title[].value] | join("")), url, snippet: ([.extract[].value] | join("")), source}'

Quirks

• Bold segment encoding: title and extract are arrays of {value, is_bold} segments. The skill concatenates value fields verbatim. If you want HTML-style highlighting you can post-process the raw JSON.
• Coverage is uneven. Best for popular sites and high-traffic content. For obscure technical queries the index may have nothing — pair with another provider.
• Be gentle on rate. The project is volunteer-run with no documented rate limits. Cache aggressively in agent harnesses.

Pricing

Free. No quota. No credit card.

See also

• Pipelines (rg / jq recipes)
• Multi-provider strategies — pair with duckduckgo for first-pass coverage
• Result schema

openai

OpenAI hosted web search via the Responses API + web_search tool. Model-mediated only — there is no raw-search alternative.

At a glance

Important: no raw search alternative

OpenAI does not offer any raw search endpoint. The legacy /v1/search was deprecated in 2022. SearchGPT remained a consumer prototype. ChatGPT Search is a consumer feature. Confirmed against current platform docs and the official Python/Node SDKs.

The Responses API + web_search tool is the only path. The deprecated gpt-4o-search-preview and gpt-4o-mini-search-preview Chat Completions models shut down 2026-07-23.

If you need model-less search, use tavily, brave, serper, or perplexity instead.

When to use

Setup

1. Get an API key from platform.openai.com/api-keys. 2. Make sure your project has access to a model that supports the web_search tool — any GPT-5.x except nano, GPT-4.1, GPT-4o, o3, o4-mini. 3. Config:

   {
     "providers": {
       "openai": {
         "apiKey": "sk-proj-...",
         "model": "gpt-5.4-mini",
         "searchContextSize": "medium",
         "mode": "live"
       }
     }
   }

4. Env: OPENAI_API_KEY.

Optional config fields:

• model (default gpt-5.4-mini) — see supported list above. Use gpt-5.5 or gpt-5.5-pro for higher synthesis quality at higher cost.
• searchContextSize — low / medium / high
• mode — live (default) or cached (sets external_web_access: false)
• userLocation — {country, region, city, timezone}
• maxToolCalls — integer, caps total tool invocations per request

Request

POST https://api.openai.com/v1/responses
Authorization: Bearer <key>
Content-Type: application/json

{
  "model": "gpt-5.4-mini",
  "input": "<query>",
  "tools": [{
    "type": "web_search",
    "external_web_access": true,
    "search_context_size": "medium",
    "filters": {
      "allowed_domains": ["docs.python.org", "..."]   ; up to 100
      ; OR
      "blocked_domains": ["spam.com"]                  ; up to 100
    },
    "user_location": {
      "type": "approximate",
      "country": "US",
      "region": "California",
      "city": "San Francisco",
      "timezone": "America/Los_Angeles"
    }
  }],
  "tool_choice": "required",
  "max_tool_calls": 5
}

Response

{
  "output": [
    {
      "type": "web_search_call",
      "id": "ws_...",
      "status": "completed",
      "action": {
        "type": "search",
        "queries": ["..."],
        "sources": [{"type": "url", "url": "https://..."}]
      }
    },
    {
      "type": "message",
      "role": "assistant",
      "content": [{
        "type": "output_text",
        "text": "Synthesized answer.",
        "annotations": [
          {
            "type": "url_citation",
            "url": "https://...",
            "title": "...",
            "start_index": 2606,
            "end_index": 2758
          }
        ]
      }]
    }
  ],
  "usage": {"total_tokens": 1370}
}

Field mapping (canonical schema)

Same logic as xai.md — walks output[] for annotations first, falls back to web_search_call.action.sources[], then top-level citations[].

Domain filtering

Native structured field — allowed_domains and blocked_domains, up to 100 entries each. Mutually exclusive.

python3 scripts/web-search --provider openai --include docs.openai.com,platform.openai.com "..."

baseUrl override

{"providers": {"openai": {
  "apiKey": "sk-...",
  "model": "gpt-5.4-mini",
  "baseUrl": "https://my-azure.openai.azure.com/openai/v1/responses?api-version=2026-04-01"
}}}

Common patterns:

• Azure-hosted OpenAI: https://<endpoint>.openai.azure.com/openai/v1/responses?api-version=...
• Corporate gateway: https://llm-gw.internal/openai/v1/responses
• Local mock for tests: http://localhost:8787/openai/v1/responses

The skill preserves any pre-existing query parameters on the override URL. See `../base-urls.md`.

curl recipe (Python-less fallback)

curl -fsSL -X POST "https://api.openai.com/v1/responses" \
  -H "Authorization: Bearer ${OPENAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.4-mini",
    "input": "what was a positive news story from today",
    "tools": [{
      "type": "web_search",
      "search_context_size": "medium",
      "external_web_access": true
    }],
    "tool_choice": "required",
    "max_tool_calls": 3
  }' \
  | jq '.output[] | select(.type=="message") | .content[].annotations[]? | {title, url}'

Quirks

• Model is mandatory. No raw search endpoint.
• `gpt-4o-mini` and `gpt-4.1-mini` are billed at a fixed 8,000 input tokens per web_search call regardless of actual context size.
• `external_web_access: false` uses cached/snapshot data instead of live web. The legacy web_search_preview tool ignores this flag (always live).
• Domain filters up to 100 entries — most generous in this catalog.
• Set `max_tool_calls` to cap runaway agentic search loops. Default is unlimited.
• The deprecated gpt-4o-search-preview and gpt-4o-mini-search-preview Chat Completions models shut down 2026-07-23.

Pricing

$10 per 1,000 web_search tool calls plus standard model token costs. The deprecated preview models cost $25/1k calls but had free search content tokens — not relevant after the 2026-07-23 shutdown.

See openai.com/api/pricing for current rates.

See also

• Pipelines, Multi-provider, Result schema, baseUrl
• For raw-search alternatives: perplexity.md, tavily.md, brave.md, serper.md

perplexity

Perplexity — two distinct endpoints. By default the skill uses the pure `/search` endpoint (no model required). If you set a model in config, the skill switches to Sonar chat completions (model required, generates an answer with built-in citations).

At a glance

The skill defaults to pure search because it is cheaper, faster, and LLM-neutral.

When to use

Setup

1. Sign up at perplexity.ai/account/api. 2. Generate an API key (pplx-…). 3. Config (pure mode):

   {"providers": {"perplexity": {"apiKey": "pplx-..."}}}

4. Config (sonar mode):

   {"providers": {"perplexity": {"apiKey": "pplx-...", "model": "sonar-pro"}}}

5. Env: PERPLEXITY_API_KEY (also PPLX_API_KEY).

Request — pure /search

POST https://api.perplexity.ai/search
Authorization: Bearer <key>
Content-Type: application/json

{
  "query": "<query>",                          ; or string[] for multi-query, max 5
  "max_results": 10,                            ; 1–20
  "max_tokens_per_page": 4096,                  ; per-result content extraction limit
  "country": "US",                              ; ISO 3166-1 alpha-2, optional
  "search_recency_filter": "week",              ; hour|day|week|month|year, optional
  "search_domain_filter": ["docs.example.com"]  ; or ["-spam.example.com"] for blocked
}

The skill exposes country and recency via the provider config block (not the CLI yet — extend in a fork if needed).

Request — Sonar chat completions

POST https://api.perplexity.ai/chat/completions
Authorization: Bearer <key>
Content-Type: application/json

{
  "model": "sonar-pro",
  "messages": [{"role": "user", "content": "<query>"}],
  "search_domain_filter": ["..."],
  "search_recency_filter": "week"
}

Response — pure /search

{
  "id": "uuid",
  "results": [
    {
      "title": "...",
      "url": "https://...",
      "snippet": "Pre-extracted content snippet...",
      "date": "2025-09-15",
      "last_updated": "2025-10-01"
    }
  ]
}

Response — Sonar

{
  "id": "...",
  "model": "sonar-pro",
  "choices": [{"message": {"role": "assistant", "content": "Generated answer..."}}],
  "citations": ["https://...", "https://..."],
  "search_results": [
    {"title": "...", "url": "...", "snippet": "...", "date": "...", "source": "web"}
  ]
}

The skill normalizes both shapes — pure results[] first, falling back to search_results[], then citations[] URLs.

Field mapping (canonical schema)

Domain filtering

Native structured field. Blocked domains are negated with - prefix.

python3 scripts/web-search --provider perplexity --include docs.python.org "asyncio"
python3 scripts/web-search --provider perplexity --exclude reddit.com,medium.com "..."

Mix-and-match is not allowed in a single call (use one or the other).

baseUrl override

Override the resolved endpoint (whichever mode you use). The skill picks the correct default based on model presence; the override fully replaces the default.

{"providers": {"perplexity": {
  "apiKey": "pplx-...",
  "baseUrl": "https://eu.api.perplexity.ai/search"
}}}

For Sonar mode override:

{"providers": {"perplexity": {
  "apiKey": "pplx-...",
  "model": "sonar-pro",
  "baseUrl": "https://my-perplexity-proxy.internal/chat/completions"
}}}

See `../base-urls.md`.

curl recipe (Python-less fallback)

Pure /search

curl -fsSL -X POST "https://api.perplexity.ai/search" \
  -H "Authorization: Bearer ${PERPLEXITY_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "open source vector database 2026",
    "max_results": 10,
    "search_recency_filter": "month"
  }' \
  | jq '.results[] | {title, url, snippet, date}'

Sonar

curl -fsSL -X POST "https://api.perplexity.ai/chat/completions" \
  -H "Authorization: Bearer ${PERPLEXITY_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "sonar-pro",
    "messages": [{"role": "user", "content": "Compare Pinecone vs Weaviate vs Qdrant"}]
  }' \
  | jq '{answer: .choices[0].message.content, sources: .citations}'

Quirks

• Two endpoints, one provider name. The skill picks based on whether model is set in config. Switching modes per-call requires editing config (no CLI flag).
• Recency filter cannot combine with date-range filters. If you set both, Perplexity returns an error.
• `country` filter is ISO 3166-1 alpha-2 ("US", "GB"). Two characters.
• Pricing: pure /search is $5/1k requests with no token cost. Sonar is per-token + per-request.
• Rate limit: 50 QPS flat on pure /search regardless of tier.

Pricing

• Pure /search: $5 per 1,000 requests. No token charges.
• Sonar: tier-based per-token + per-request. See docs.perplexity.ai/docs/getting-started/pricing.

See also

• Pipelines, Multi-provider, Result schema, baseUrl

serper

Serper — Google's organic search results via a paid proxy. Cheap, fast, and returns the actual Google ranking.

At a glance

When to use

Setup

1. Sign up at serper.dev. 2. Get the default API key from the dashboard. 3. Config:

   {"providers": {"serper": {"apiKey": "0123456789abcdef..."}}}

Or env: SERPER_API_KEY.

Free-tier credits (2,500) are usually enough for an experiment or small production workload.

Request

POST https://google.serper.dev/search
X-API-KEY: <key>
Content-Type: application/json

{
  "q": "<query>",
  "num": 10
}

Response

{
  "organic": [
    {
      "title": "...",
      "link": "https://...",
      "snippet": "...",
      "position": 1,
      "date": "2 days ago"
    }
  ],
  "knowledgeGraph": {...},
  "answerBox": {...},
  "peopleAlsoAsk": [...]
}

Field mapping (canonical schema)

The skill ignores knowledgeGraph, answerBox, and peopleAlsoAsk — use the raw response if you need them.

Domain filtering

Serper does not accept structured domain filters. The skill appends site: / -site: operators to the q field.

python3 scripts/web-search --provider serper --exclude reddit.com,medium.com "..."

baseUrl override

{"providers": {"serper": {"apiKey": "...", "baseUrl": "https://serper-proxy.internal/search"}}}

curl recipe (Python-less fallback)

curl -fsSL -X POST "https://google.serper.dev/search" \
  -H "X-API-KEY: ${SERPER_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"q": "elasticsearch alternatives", "num": 10}' \
  | jq '.organic[] | {title, url: .link, snippet}'

Quirks

• `date` is a human-readable string ("2 days ago"). The skill copies it verbatim to publishedAt. Parse downstream if you need a real date.
• Includes Google's structured features (knowledge graph, answer box, "people also ask") in the raw response. Inspect the <provider>-raw.json file for these.

Pricing

See serper.dev/pricing. One of the cheapest paid providers in the catalog.

See also

• Pipelines, Multi-provider, Result schema

tavily

Tavily — search API designed specifically for LLM agents. Returns clean, pre-extracted snippets ideal for downstream model consumption.

At a glance

When to use

Setup

1. Sign up at app.tavily.com/sign-in. 2. The dashboard generates a default API key (tvly-…). Create more on the API Keys page. 3. Config:

   {"providers": {"tavily": {"apiKey": "tvly-..."}}}

Or env: TAVILY_API_KEY.

Request

POST https://api.tavily.com/search
Authorization: Bearer <key>
Content-Type: application/json

{
  "query": "<query>",
  "max_results": 10,
  "include_domains": [...],     ; optional
  "exclude_domains": [...]       ; optional
}

The skill clamps max_results to 20.

Response

{
  "query": "...",
  "results": [
    {
      "title": "...",
      "url": "https://...",
      "content": "Pre-extracted snippet, cleaned for LLM consumption.",
      "score": 0.92,
      "published_date": "2025-09-12"
    }
  ],
  "response_time": 0.83
}

Field mapping (canonical schema)

Domain filtering

Native structured fields (include_domains, exclude_domains).

python3 scripts/web-search --provider tavily --include docs.python.org "asyncio"

baseUrl override

{"providers": {"tavily": {"apiKey": "tvly-...", "baseUrl": "https://eu.api.tavily.com/search"}}}

Useful for regional mirrors and corporate gateways. See `../base-urls.md`.

curl recipe (Python-less fallback)

curl -fsSL -X POST "https://api.tavily.com/search" \
  -H "Authorization: Bearer ${TAVILY_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"query": "rust async runtime comparison", "max_results": 10}' \
  | jq '.results[] | {title, url, content}'

Quirks

• The content field is already cleaned (no HTML tags, no boilerplate). Tavily extracts page content per result and trims to a reasonable length.
• The free tier's 1k/month resets monthly — keep an eye on it for high-volume agents.

Pricing

Tiered — see tavily.com/pricing. Free tier covers most light usage.

See also

• Pipelines, Multi-provider, Result schema

xai

xAI Grok web search via the Responses API + web_search tool. Model-mediated only — Grok decides when, how, and how often to search. There is no raw-search alternative.

At a glance

Important: no raw search alternative

The deprecated search_parameters field on /v1/chat/completions (Live Search) returns HTTP 410 Gone as of Jan 2026. The Responses API + web_search tool is the only supported path. Confirmed via xAI docs and SDK source.

If you need model-less search, use tavily, brave, serper, or perplexity instead.

When to use

Setup

1. Sign up at console.x.ai. 2. Generate an API key (xai-…). 3. Add a payment method — every request is billed. 4. Config:

   {
     "providers": {
       "xai": {
         "apiKey": "xai-...",
         "model": "grok-4.3"
       }
     }
   }

5. Env: XAI_API_KEY.

model is optional in config — the skill defaults to grok-4.3. Set it explicitly if you want a specific Grok variant (e.g. grok-4.20-reasoning for agentic search).

Request

POST https://api.x.ai/v1/responses
Authorization: Bearer <key>
Content-Type: application/json

{
  "model": "grok-4.3",
  "input": "<query>",
  "tools": [{
    "type": "web_search",
    "filters": {
      "allowed_domains": ["docs.example.com"]   ; max 5
      ; OR
      "excluded_domains": ["spam.example.com"]   ; max 5
    }
  }],
  "tool_choice": "required"
}

The skill caps domain filters at 5 entries (xAI's hard limit).

Response

OpenAI-compatible Responses API shape:

{
  "output": [
    {
      "type": "web_search_call",
      "id": "ws_...",
      "action": {
        "type": "search",
        "queries": ["..."],
        "sources": [{"type": "url", "url": "https://..."}]
      }
    },
    {
      "type": "message",
      "role": "assistant",
      "content": [{
        "type": "output_text",
        "text": "Synthesized answer with [[1]](https://...) citations.",
        "annotations": [
          {
            "type": "url_citation",
            "url": "https://...",
            "title": "...",
            "start_index": 0,
            "end_index": 30
          }
        ]
      }]
    }
  ],
  "citations": ["https://..."],
  "usage": {"input_tokens": 32, "output_tokens": 9, "total_tokens": 151}
}

Field mapping (canonical schema)

The skill walks output[]: 1. Annotations in output[].content[].annotations[] (preferred) — gives title, url, and cited_text. 2. Sources in output[].action.sources[] (fallback) — gives url and title. 3. Top-level `citations[]` (last fallback) — bare URL list.

Domain filtering

Native structured field. Max 5 entries each (allowed_domains or excluded_domains). Mutually exclusive.

python3 scripts/web-search --provider xai --include docs.x.ai,grokipedia.com "..."

baseUrl override

{"providers": {"xai": {
  "apiKey": "xai-...",
  "model": "grok-4.3",
  "baseUrl": "https://xai-proxy.internal/v1/responses"
}}}

The override URL must accept the standard Responses API body shape. See `../base-urls.md`.

curl recipe (Python-less fallback)

curl -fsSL -X POST "https://api.x.ai/v1/responses" \
  -H "Authorization: Bearer ${XAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "grok-4.3",
    "input": "latest news about EU AI Act",
    "tools": [{"type": "web_search"}],
    "tool_choice": "required"
  }' \
  | jq '.output[] | select(.type=="message") | .content[].annotations[]? | {title, url}'

To suppress inline citation markers ([[1]](...)) in the answer text:

curl ... -d '{... "include": ["no_inline_citations"], ...}'

Quirks

• Model is mandatory. No raw search.
• Domain filters limited to 5 entries — fewer than OpenAI (100) or Anthropic (no documented limit).
• `web_search` and `x_search` are separate tools. Pair them in the same tools array if you want both general web and X/Twitter posts:

  "tools": [{"type": "web_search"}, {"type": "x_search"}]

• Model decides search count. No direct way to force exactly N searches.
• The deprecated Live Search /v1/chat/completions path is HTTP 410 Gone — do not use.

Pricing

$5 per 1,000 web_search tool calls plus model token costs. See docs.x.ai/docs/models.

See also

• Pipelines, Multi-provider, Result schema, baseUrl
• For raw-search alternatives: perplexity.md, tavily.md, brave.md

z-ai

Z.ai (Zhipu) web search endpoint. Best coverage for Chinese-language queries and Chinese-region content.

At a glance

When to use

Setup

1. Sign up at z.ai. 2. Get an API key from the developer console. 3. Config:

   {"providers": {"z-ai": {"apiKey": "..."}}}

Env: Z_AI_API_KEY (also ZAI_API_KEY, ZHIPU_API_KEY).

Request

POST https://api.z.ai/api/paas/v4/web_search
Authorization: Bearer <key>
Content-Type: application/json

{
  "search_engine": "search-prime",
  "search_query": "<query>",
  "count": 10,
  "search_domain_filter": "example.com"   ; optional, single domain only
}

Response

{
  "search_result": [
    {
      "title": "...",
      "link": "https://...",
      "content": "...",
      "media": "Example",
      "publish_date": "2025-09-01"
    }
  ]
}

Field mapping (canonical schema)

Domain filtering

Single domain only. Z.ai's search_domain_filter is a string field, not an array. The skill takes the first allowed domain and ignores the rest. Blocked domains are not supported.

python3 scripts/web-search --provider z-ai --include zhihu.com "transformer 优化"

baseUrl override

{"providers": {"z-ai": {"apiKey": "...", "baseUrl": "https://zai-proxy.internal/web_search"}}}

You can also override searchEngine in config to use a different upstream search engine name (default search-prime):

{"providers": {"z-ai": {"apiKey": "...", "searchEngine": "search-pro"}}}

See `../base-urls.md`.

curl recipe (Python-less fallback)

curl -fsSL -X POST "https://api.z.ai/api/paas/v4/web_search" \
  -H "Authorization: Bearer ${Z_AI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"search_engine": "search-prime", "search_query": "ChatGLM 开源", "count": 10}' \
  | jq '.search_result[] | {title, url: .link, snippet: .content, source: .media}'

Quirks

• Single-domain filter only. If you pass --include a.com,b.com, the skill uses a.com and silently drops b.com. Use a different provider for multi-domain restriction.
• No blocked-domain support at the API level. Use --exclude only with other providers, or filter the result file post-hoc with jq.
• search_engine is hardcoded to search-prime. Z.ai may expose other engines (search-pro, etc.) — override via the searchEngine config field.

Pricing

See Z.ai's developer dashboard for current tiers. Pricing is usage-based; tiers vary.

See also

• Pipelines, Multi-provider, Result schema