Data Scraper Agent
- 1.4k installs
- 238k repo stars
- Updated August 5, 2026
- affaan-m/ecc
This is a copy of data-scraper-agent by affaan-m - installs and ranking accrue to the original listing.
data-scraper-agent is a Claude Code skill that generates a scheduled Python data-collection agent with LLM enrichment and database exports for developers monitoring public web sources.
About
data-scraper-agent is a community ECC skill that builds production-ready, AI-powered data collection agents for public sources such as job boards, prices, news, GitHub repos, sports scores, and event listings. The generated stack uses Python, Gemini Flash for free LLM enrichment, GitHub Actions for scheduled runs at no hosting cost, and storage in Notion, Google Sheets, or Supabase. The agent scrapes on a schedule, enriches results with an LLM, persists output, and improves from user feedback over time. Developers reach for data-scraper-agent when they want to monitor or track public data without writing a scraper from scratch or paying for dedicated hosting infrastructure.
- Generates a full three-layer agent (COLLECT → ENRICH → STORE) in one prompt
- Uses free stack: Python, Gemini Flash, GitHub Actions, Notion/Sheets/Supabase
- Enriches scraped data with LLM classification, summarization and scoring
- Stores results automatically and improves from user feedback over time
- Runs on a free cron schedule with zero hosting cost
Data Scraper Agent by the numbers
- 1,416 all-time installs (skills.sh)
- +93 installs in the week ending Aug 5, 2026 (Skillselion tracking)
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/affaan-m/ecc --skill data-scraper-agentAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 1.4k |
|---|---|
| repo stars | ★ 238k |
| Last updated | August 5, 2026 |
| Repository | affaan-m/ecc ↗ |
How do you build a scheduled public data scraper for free?
Instantly generate a complete, scheduled AI data-collection agent for any public source without writing the scraper from scratch.
Who is it for?
Developers who need recurring public-data monitoring across job boards, prices, news, or repos without paid hosting.
Skip if: Developers scraping authenticated private APIs, real-time sub-second feeds, or workloads that cannot run on GitHub Actions schedules.
When should I use this skill?
A developer asks to monitor, scrape, or track public websites, APIs, jobs, prices, news, repos, or sports scores on a schedule.
What you get
Python scraper scripts, GitHub Actions workflow, LLM enrichment pipeline, and Notion/Sheets/Supabase storage integration.
- Python scraper agent
- GitHub Actions schedule
- Storage integration config
Files
データスクレイパーエージェント
任意のパブリックデータソース用の本番環境対応、AI搭載データ収集エージェントを構築。 スケジュールで実行され、無料LLMで結果を豊かにし、データベースに保存し、時間とともに改善されます。
スタック：Python · Gemini Flash（無料） · GitHub Actions（無料） · Notion / Sheets / Supabase
アクティベーション時期
- ユーザーが任意のパブリックWebサイトまたはAPIをスクレイプまたは監視したい場合
- ユーザーが「チェックするボットを構築」「Xを監視」「データを収集」と言う
- ユーザーがジョブ、価格、ニュース、リポ、スポーツスコア、イベント、リストを追跡したい場合
- ユーザーがホスティング用に支払わずにデータ収集を自動化する方法を尋ねる
- ユーザーが決定に基づいて時間とともにより スマートになるエージェントを望む
コアコンセプト
3つのレイヤー
すべてのデータスクレイパーエージェントには3つのレイヤーがあります：
COLLECT → ENRICH → STORE
│ │ │
Scraper AI (LLM) Database
runs on scores/ Notion /
schedule summarises Sheets /
& classifies Supabase無料スタック
| Layer | Tool | Why |
|---|---|---|
| COLLECT | Playwright/BeautifulSoup | 無料のオープンソーススクレイピング |
| ENRICH | Gemini Flash | 無料で高速LLM |
| STORE | Supabase / Sheets | 無料データベースとスプレッドシート |
| SCHEDULE | GitHub Actions | 無料クロンジョブ |
ワークフロー
1. ソースを定義 - どこからスクレイプするか、何を抽出するか 2. スクレイパーを構築 - BeautifulSoup または Playwright ベースのコレクタ 3. LLMを構成 - Gemini Flash でテキストをスコア付け/要約/分類 4. ストレージを設定 - Notion、Sheets、Supabase のいずれか 5. GitHub Actions を設定 - 毎日/毎週実行するスケジュール 6. フィードバックループを追加 - ユーザーの判断から学習
例
- ジョブボード監視：新しい公開
Related skills
Agent BrowserGive their coding agent reliable, high-fidelity control over any website or Electron desktop app.591k39.6k
Lark ApprovalLet their AI coding agent create, read, update, and approve items in Lark (Feishu) approval workflows without leaving the coding environment.494kLark EventHandle Feishu/Lark bot events, webhooks, and subscription callbacks in agent-driven backend code.493kLark Workflow Meeting SummaryAutomatically generate structured meeting summaries and action items from Lark/Feishu calls without manual note-taking.492kLark Workflow Standup ReportAutomatically generate and post daily standup reports in Lark/Feishu from their workflow and activity data.492k
Twitter AutomationAutomate X/Twitter posts, scheduling, and engagement via X API integration441k669How it compares
Choose data-scraper-agent over manual script snippets when you need a full scheduled agent with LLM enrichment and storage wiring, not a one-off curl parser.
FAQ
What stack does data-scraper-agent generate?
data-scraper-agent generates Python scrapers scheduled via GitHub Actions, enriches output with Gemini Flash, and stores results in Notion, Google Sheets, or Supabase. The workflow is designed to run without paid hosting.
What public sources can data-scraper-agent target?
data-scraper-agent targets public sources including job boards, price pages, news sites, GitHub repositories, sports scores, events, and general list pages that can be scraped or polled on a schedule.