Scrapingant
Pull live page content and structured extracts through ScrapingAnt so your agent can research markets and competitors without brittle curl scripts.
Overview
com.mcparmory/scrapingant is a MCP server for the Idea phase that scrapes webpages with JavaScript and CAPTCHA support and returns structured data through ScrapingAnt.
What is this MCP server?
- Scrape webpages via ScrapingAnt with JavaScript rendering handled in the service
- CAPTCHA handling path for pages that block headless fetches
- Return structured data extracts suitable for downstream analysis in the agent
- stdio MCP via mcparmory-scrapingant 1.0.1 (uvx) or Docker ghcr.io/mcparmory/scrapingant:1.0.1
- Dual PyPI and OCI packages from the MCP Armory registry
- Server version 1.0.1
- 2 packages: PyPI mcparmory-scrapingant and OCI ghcr.io/mcparmory/scrapingant:1.0.1
- stdio transport only
Community signal: 25 GitHub stars.
What problem does it solve?
Validation research stalls when target sites need a real browser or CAPTCHA solve and your agent only has basic HTTP fetches.
Who is it for?
Solo builders doing fast opportunity research on JS-heavy sites who already use or will trial ScrapingAnt credits.
Skip if: Large-scale crawling pipelines, strictly on-prem scraping, or teams that cannot use a third-party scraping API for compliance reasons.
What do I get? / Deliverables
Your agent can request rendered pages and structured extracts in one tool chain so competitor and market research stays inside the IDE session.
- Fetched page content and structured extracts usable in agent summaries
- Repeatable scrape requests logged in the MCP session
- Registry-aligned server at version 1.0.1
Recommended MCP Servers
Journey fit
Competitive and market research during Idea happens before you commit to a build; scripted fetching with JS rendering belongs at the research shelf first. Research is where you harvest public web signals—pricing pages, listings, docs—that JavaScript-heavy sites block from simple HTTP clients.
How it compares
Hosted anti-bot scraping MCP bridge, not a self-hosted Playwright skill or a static CSV import tool.
Common Questions / FAQ
Who is com.mcparmory/scrapingant for?
Builders and researchers who want Claude Code or Cursor to fetch and structure web pages during idea validation without maintaining their own browser automation stack.
When should I use com.mcparmory/scrapingant?
Use it in Idea research (or early Validate) when you need rendered DOM content, CAPTCHA bypass via ScrapingAnt, or structured fields from public competitor and market pages.
How do I add com.mcparmory/scrapingant to my agent?
Add a stdio MCP entry pointing to uvx mcparmory-scrapingant 1.0.1 or the Docker image ghcr.io/mcparmory/scrapingant:1.0.1, and configure your ScrapingAnt API key per client docs.