Data Scraper Agent
- 5.3k installs
- 238k repo stars
- Updated August 5, 2026
- affaan-m/everything-claude-code
data-scraper-agent is a skill for building fully automated AI-powered data collection agents that scrape public sources, enrich with Gemini Flash, and store results on a free schedule.
About
data-scraper-agent builds production-ready automated AI data collection agents for any public source including job boards, prices, news, GitHub, and sports scores. The three-layer architecture follows COLLECT, ENRICH, STORE: Playwright or BeautifulSoup scrapers run on schedule, Gemini Flash scores and classifies extracted text, and results land in Notion, Google Sheets, or Supabase. GitHub Actions provides free cron scheduling with no paid hosting required. Workflow steps define the source and fields, implement the scraper collector, configure Gemini enrichment, set storage, wire the schedule, and add a feedback loop from user judgments. Activate when users want to scrape or monitor public websites, track jobs, prices, news, repos, sports scores, events, or lists, or ask how to automate collection without paying for hosting. Examples include job board monitoring for new postings. The stack is Python with open-source scraping, free LLM enrichment, and free tier storage backends.
- Three-layer COLLECT, ENRICH, STORE pipeline with scraper, Gemini Flash, and database storage.
- Free stack: Playwright or BeautifulSoup, Gemini Flash, GitHub Actions cron, Notion, Sheets, Supabase.
- Six-step workflow from source definition through scraper, LLM config, storage, schedule, and feedback loop.
- Targets job boards, prices, news, GitHub repos, sports scores, events, and public list monitoring.
- Runs on GitHub Actions with no paid hosting required for scheduled collection.
Data Scraper Agent by the numbers
- 5,318 all-time installs (skills.sh)
- +252 installs in the week ending Aug 4, 2026 (Skillselion tracking)
- Ranked #67 of 2,715 Automation & Workflows skills by installs in the Skillselion catalog
- Security screen: MEDIUM risk (skills.sh audit)
- Data as of Aug 5, 2026 (Skillselion catalog sync)
data-scraper-agent capabilities & compatibility
- Capabilities
- playwright or beautifulsoup public source scrapi · gemini flash scoring summarization and classific · notion sheets supabase result storage · github actions cron scheduling · user feedback loop for agent improvement · source definition and field extraction workflow
- Works with
- playwright · github · supabase · notion · openai
- Use cases
- web scraping · orchestration · data analysis
- Runs
- Remote server
- Pricing
- Free
What data-scraper-agent says it does
COLLECT → ENRICH → STORE
スタック：Python · Gemini Flash（無料） · GitHub Actions（無料） · Notion / Sheets / Supabase
ユーザーが任意のパブリックWebサイトまたはAPIをスクレイプまたは監視したい場合
npx skills add https://github.com/affaan-m/everything-claude-code --skill data-scraper-agentAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 5.3k |
|---|---|
| repo stars | ★ 238k |
| Security audit | 2 / 3 scanners passed |
| Last updated | August 5, 2026 |
| Repository | affaan-m/everything-claude-code ↗ |
How do I monitor and collect public web data on a schedule without paid hosting while enriching and storing results automatically?
Build scheduled public-data scrapers enriched with Gemini Flash and stored in Notion, Sheets, or Supabase via GitHub Actions.
Who is it for?
Teams tracking public job boards, prices, news, repos, sports scores, or lists who want free scheduled collection with LLM enrichment.
Skip if: Skip for private authenticated APIs without scraping, one-off manual lookups, or tasks unrelated to automated public data collection.
When should I use this skill?
User wants to scrape or monitor a public website or API, build a check bot, collect data, or automate tracking without paid hosting.
What you get
Scheduled scraper, Gemini-enriched records, configured storage, GitHub Actions cron, and a feedback loop that improves over time.
- python scraper script
- GitHub Actions workflow
- database storage integration
Files
データスクレイパーエージェント
任意のパブリックデータソース用の本番環境対応、AI搭載データ収集エージェントを構築。 スケジュールで実行され、無料LLMで結果を豊かにし、データベースに保存し、時間とともに改善されます。
スタック：Python · Gemini Flash（無料） · GitHub Actions（無料） · Notion / Sheets / Supabase
アクティベーション時期
- ユーザーが任意のパブリックWebサイトまたはAPIをスクレイプまたは監視したい場合
- ユーザーが「チェックするボットを構築」「Xを監視」「データを収集」と言う
- ユーザーがジョブ、価格、ニュース、リポ、スポーツスコア、イベント、リストを追跡したい場合
- ユーザーがホスティング用に支払わずにデータ収集を自動化する方法を尋ねる
- ユーザーが決定に基づいて時間とともにより スマートになるエージェントを望む
コアコンセプト
3つのレイヤー
すべてのデータスクレイパーエージェントには3つのレイヤーがあります：
COLLECT → ENRICH → STORE
│ │ │
Scraper AI (LLM) Database
runs on scores/ Notion /
schedule summarises Sheets /
& classifies Supabase無料スタック
| Layer | Tool | Why |
|---|---|---|
| COLLECT | Playwright/BeautifulSoup | 無料のオープンソーススクレイピング |
| ENRICH | Gemini Flash | 無料で高速LLM |
| STORE | Supabase / Sheets | 無料データベースとスプレッドシート |
| SCHEDULE | GitHub Actions | 無料クロンジョブ |
ワークフロー
1. ソースを定義 - どこからスクレイプするか、何を抽出するか 2. スクレイパーを構築 - BeautifulSoup または Playwright ベースのコレクタ 3. LLMを構成 - Gemini Flash でテキストをスコア付け/要約/分類 4. ストレージを設定 - Notion、Sheets、Supabase のいずれか 5. GitHub Actions を設定 - 毎日/毎週実行するスケジュール 6. フィードバックループを追加 - ユーザーの判断から学習
例
- ジョブボード監視：新しい公開
Related skills
Agent BrowserGive their coding agent reliable, high-fidelity control over any website or Electron desktop app.591k39.6k
Lark ApprovalLet their AI coding agent create, read, update, and approve items in Lark (Feishu) approval workflows without leaving the coding environment.494kLark EventHandle Feishu/Lark bot events, webhooks, and subscription callbacks in agent-driven backend code.493kLark Workflow Meeting SummaryAutomatically generate structured meeting summaries and action items from Lark/Feishu calls without manual note-taking.492kLark Workflow Standup ReportAutomatically generate and post daily standup reports in Lark/Feishu from their workflow and activity data.492k
Twitter AutomationAutomate X/Twitter posts, scheduling, and engagement via X API integration441k669Forks & variants (1)
Data Scraper Agent has 1 known copy in the catalog totaling 1.4k installs. They canonicalize to this original listing.
- affaan-m - 1.4k installs
How it compares
Choose data-scraper-agent over manual scraper scripts when you need scheduled runs, LLM enrichment, and database storage wired together in one scaffold.
FAQ
What is the three-layer architecture?
COLLECT runs Playwright or BeautifulSoup on schedule, ENRICH uses Gemini Flash to score and classify text, STORE saves to Notion, Sheets, or Supabase.
How does scheduling work without paid hosting?
GitHub Actions runs the scraper on a daily or weekly cron at no hosting cost.
When should I activate this skill?
When the user wants to scrape or monitor public websites, track jobs, prices, news, repos, or lists, or automate collection for free.
Is Data Scraper Agent safe to install?
skills.sh reports 2 of 3 security scanners passed. Review the Security Audits panel on this page before installing in production.