Scrape

Name: Scrape
Author: davila7

davila7/claude-code-templates

13 installs
30.1k repo stars
Updated August 4, 2026
davila7/claude-code-templates

Scrape any webpage as clean markdown via the Bright Data Web Unlocker API, bypassing bot detection and CAPTCHA.

About

Fetches webpage content as clean markdown through Bright Data's Web Unlocker API with automatic bot and CAPTCHA bypass. A developer uses it when scraping pages that block ordinary requests. Requires BRIGHTDATA_API_KEY and unlocker zone.

Returns clean markdown from any URL
Bypasses bot detection and CAPTCHA via Bright Data

Scrape by the numbers

13 all-time installs (skills.sh)
Ranked #1,436 of 2,715 Automation & Workflows skills by installs in the Skillselion catalog
Data as of Aug 5, 2026 (Skillselion catalog sync)

npx skills add https://github.com/davila7/claude-code-templates --skill scrape

Add your badge

Show developers this skill is listed on Skillselion. Paste this into your README.

[![Listed on Skillselion](https://skillselion.com/badge/skills/davila7/claude-code-templates/scrape.svg)](https://skillselion.com/skills/davila7/claude-code-templates/scrape)

Installs	13
repo stars	★ 30.1k
Last updated	August 4, 2026
Repository	davila7/claude-code-templates ↗

What it does

Scrape any webpage as clean markdown via the Bright Data Web Unlocker API, bypassing bot detection and CAPTCHA.

Files

SKILL.mdMarkdownGitHub ↗

Bright Data - Web Scraper

Scrape any webpage and get clean markdown content using Bright Data's Web Unlocker API. Automatically bypasses bot detection and CAPTCHA.

Setup

1. Get your API Key: Get a key from Bright Data Dashboard.

2. Create a Web Unlocker zone: Create a zone at brightdata.com/cp by clicking "Add" (top-right), selecting "Unlocker zone".

3. Set environment variables:

export BRIGHTDATA_API_KEY="your-api-key"
export BRIGHTDATA_UNLOCKER_ZONE="your-zone-name"

Usage

bash scripts/scrape.sh "url"

Parameters:

url (required): The webpage URL to scrape

Examples:

# Scrape a news article
bash scripts/scrape.sh "https://example.com/article"

# Scrape a product page
bash scripts/scrape.sh "https://shop.example.com/product/123"

Output Format

Returns clean markdown content extracted from the webpage:

# Page Title

Main content of the page converted to markdown format...

## Section Heading

More content...

Features

Bot Detection Bypass: Automatically handles anti-bot measures
CAPTCHA Solving: Bypasses CAPTCHA challenges
Clean Markdown: Returns well-formatted markdown content
JavaScript Rendering: Handles JavaScript-heavy pages

Dependencies

curl - For API requests

#!/bin/bash
# Bright Data Web Scraper (markdown output)

URL="$1"

if [ -z "$URL" ]; then
    echo "Usage: $0 \"url\"" >&2
    exit 1
fi

if [ -z "${BRIGHTDATA_API_KEY:-}" ]; then
    echo "Error: BRIGHTDATA_API_KEY is not set." >&2
    echo "Get a key from https://brightdata.com/cp" >&2
    exit 1
fi

if [ -z "${BRIGHTDATA_UNLOCKER_ZONE:-}" ]; then
    echo "Error: BRIGHTDATA_UNLOCKER_ZONE is not set." >&2
    echo "Create a zone at brightdata.com/cp" >&2
    exit 1
fi

# Call Bright Data API
PAYLOAD=$(jq -n \
    --arg url "$URL" \
    --arg zone "$BRIGHTDATA_UNLOCKER_ZONE" \
    '{
        url: $url,
        zone: $zone,
        format: "raw",
        data_format: "markdown"
    }')

curl -s -X POST 'https://api.brightdata.com/request' \
    -H "Authorization: Bearer $BRIGHTDATA_API_KEY" \
    -H 'Content-Type: application/json' \
    -d "$PAYLOAD"