Web Researcher

Web Researcher Skill

Production-grade live-web research bridge: search across the engine you choose, narrow to the sources you trust (lenses), read the full source (not snippets), and get citations you can verify — never fabricated, never a closed pre-synthesized garden. Runs locally/private; never invokes a browser from inside this skill.

Which web-search skill? (read this first — avoids the common mix-up)

One-liner: perplexity-research = closed engine, synthesized answer; web-researcher = your engine + trusted lenses, real verifiable sources; deep-research = the harness over both.

When To Use

• Research where your reputation is on the line — citations must be real and checkable.
• Restrict search to trusted sources via lenses (PubMed/arXiv/SEC/.gov, not random blogs).
• Read full articles — web pages, PDF/DOCX/PPTX, YouTube transcripts, Hacker News threads.
• Academic work: real papers + DOIs (academic_search), citation neighborhoods (citation_graph).
• Domain search: courts (legal_search), trials (clinical_search), macro/econ (econ_search), SEC (filing_search).
• Verify a citation before relying on it (verify_citation) or audit a whole reference list (audit_bibliography); snapshot a source to Wayback (archive_source); export a bibliography (format_bibliography).
• A direct, source-cited answer instead of a reading list; structured JSON extraction (structured_search).
• Session-tracked multi-step digs (sequential_search) with recovery + export.

When Not To Use

• Quick casual lookup, no citing needed → perplexity-research or Claude built-in search.
• Single known URL summary → gemini-url-context or web-summary.
• Interactive browser flows (login, click, form-fill) → browser / playwright.
• WebGPU/WebGL rendering validation → browser / chrome-cdp.
• Multi-agent report with adversarial verification → deep-research (it can call this skill).
• Iterative metric-optimisation experiment loops → autoresearch.
• Grounding in OUR formal ontology/KG → ontology-augment.

Connection

MCP stdio server registered as web-researcher in mcp/mcp.json. Binary resolved from $PATH (Nix-baked when [skills.research].web_researcher = true, else go install github.com/zoharbabin/web-researcher-mcp/cmd/web-researcher-mcp@latest).

# Manual registration (auto-registered at boot):
claude mcp add --scope user --transport stdio web-researcher -- web-researcher-mcp

Version gap (read this): this doc tracks upstream v1.33.0, but the **deployed

Nix binary is currently v1.2.2** (the original 8 tools: web/image/news/academic/patent/

sequential search + scrape_page + search_and_scrape). The newer tools below (domain

search, citation integrity, answer, session/memory) and the expanded lens set go live

only after bumping the flake.nix pin (webResearcherMcpPkg: version = "1.33.0",

rev v1.33.0 = commit 8ccf4c7e, refresh hash + vendorHash via

nix-prefetch-github zoharbabin web-researcher-mcp --rev v1.33.0 and the first build's

printed vendorHash) and rebuilding agentbox. Until then, **only the 8 v1.2.2 tools are

live** — run tools/list (Health Check) to confirm what the server actually exposes.

Tools (live set depends on configured providers — see notes)

Search & read

Domain search

Citation integrity & output

Session / memory / collaboration (some are opt-in, consent-gated by the operator)

docs/TOOLS.md upstream is the authoritative, CI-verified tool list + schemas.

Search Lenses (the differentiator)

Lenses restrict results to a curated set of trusted domains for a field, instead of the whole web. Built-in lenses ship with the binary (catalog at lenses://catalog): academic, academic-extended, clinical, legal, finance, government, journalism, devops, docs (+ custom JSON lenses). Reference by short name:

web_search({ query: "ML-KEM constant-time implementations", lens: "academic", count: 8 })
web_search({ query: "FDA breakthrough designation 2026", lens: "clinical" })

MCP Resources & Prompts

Live status/diagnostics the agent can read: stats://tools, stats://sessions, stats://rate-limits, stats://providers, lenses://catalog, diagnostics://errors/recent, diagnostics://health, and a large-payload artifact store research://artifact/{id}. Ready-made research prompt templates appear as / commands.

Browser Delegation (Critical)

web-researcher-mcp's tier-4 scrape (go-rod + stealth) would auto-download its own Chromium per pod. In agentbox this is disabled — a second Chromium duplicates the browsercontainer sidecar (Chrome Beta 149+, NVIDIA Vulkan, VNC :5903, MCP SSE :8931):

SCRAPER_DISABLE_BROWSER=true
CHROME_PATH=                       # empty -> tier 4 hard-fails fast

When tiers 1–3 (markdown negotiation, stealth HTTP, HTML parser) can't extract a JS-rendered page (NEEDS_BROWSER), switch to the browser skill — do not retry inside this MCP:

1. scrape_page(url)  -> tiers 1-3 ok => done ; all fail => NEEDS_BROWSER
2. browser_navigate({url}) on browser-gpu
3. browser_snapshot() / browser_evaluate(...)

One Chrome surface, one GPU allocation, one audit trail.

Required Configuration

Set at least one search provider (read from session env):

GOOGLE_CUSTOM_SEARCH_API_KEY  GOOGLE_CUSTOM_SEARCH_ID   # PSE (default)
BRAVE_API_KEY                                           # Brave
SERPER_API_KEY                                          # Serper.dev
SEARCHAPI_API_KEY                                       # SearchAPI.io
EXA_API_KEY                                             # Exa (enables `answer` + `structured_search`)
SEARXNG_URL                                             # self-hosted
EDGAR_CONTACT_EMAIL                                     # enables filing_search (SEC EDGAR)

Multi-provider routing with per-provider circuit breakers + failover:

export SEARCH_ROUTING=brave,google,serper
# or per-operation JSON:
export SEARCH_ROUTING='{"web":"brave,google","news":"brave,serper","images":"google,brave","default":"brave,google"}'

Composition Patterns

Health Check

echo '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' | web-researcher-mcp 2>/dev/null | head -1

References

• Upstream (v1.33.0): https://github.com/zoharbabin/web-researcher-mcp
• Authoritative tools: docs/TOOLS.md ; deployment: docs/DEPLOYMENT.md (upstream)
• Related skills: perplexity-research (fast closed synthesis), deep-research

(multi-agent harness), browser, gemini-url-context, web-summary, provenance-tracking, autoresearch, ontology-augment