Firecrawl Crawl
- 75.7k installs
- 561 repo stars
- Updated August 4, 2026
- firecrawl/cli
firecrawl crawl is a CLI tool that extracts content from entire websites by following links with depth and scope limits.
About
Bulk extracts content from entire websites by following links up to specified depth. Supports path filtering, concurrent extraction, depth limits, and async job handling with status monitoring.
- Bulk crawls websites following links up to specified depth and page limit
- Path filtering to scope crawls to specific sections (e.g., /docs)
- Concurrent extraction with progress tracking and async job monitoring
Firecrawl Crawl by the numbers
- 75,668 all-time installs (skills.sh)
- +840 installs in the week ending Aug 5, 2026 (Skillselion tracking)
- Ranked #10 of 550 CLI & Terminal skills by installs in the Skillselion catalog
- Security screen: MEDIUM risk (skills.sh audit)
- Data as of Aug 5, 2026 (Skillselion catalog sync)
firecrawl-crawl capabilities & compatibility
2-5 credits per page crawled
- Capabilities
- web crawling · bulk extraction · content indexing · site mapping
- Use cases
- web scraping
- Runs
- Runs locally
- Pricing
- Freemium
What firecrawl-crawl says it does
Bulk extract content from a website. Crawls pages following links up to a depth/limit.
npx skills add https://github.com/firecrawl/cli --skill firecrawl-crawlAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 75.7k |
|---|---|
| repo stars | ★ 561 |
| Security audit | 2 / 3 scanners passed |
| Last updated | August 4, 2026 |
| Repository | firecrawl/cli ↗ |
What it does
Extract content from multiple pages on a website at once.
Who is it for?
Extracting documentation sites, knowledge bases, or specific sections of large websites.
Skip if: Crawling extremely large sites (billions of pages) or heavy JavaScript-heavy dynamic applications.
When should I use this skill?
You need to extract many pages from the same website or site section.
What you get
Automatically crawl and extract all pages in a site section as structured markdown.
- JSON file with all crawled page content
Files
firecrawl crawl
Bulk extract content from a website. Crawls pages following links up to a depth/limit.
When to use
- You need content from many pages on a site (e.g., all
/docs/) - You want to extract an entire site section
- Step 4 in the workflow escalation pattern: search → scrape → map → crawl → interact
Quick start
# Crawl a docs section
firecrawl crawl "<url>" --include-paths /docs --limit 50 --wait -o .firecrawl/crawl.json
# Full crawl with depth limit
firecrawl crawl "<url>" --max-depth 3 --wait --progress -o .firecrawl/crawl.json
# Check status of a running crawl
firecrawl crawl <job-id>Options
| Option | Description |
|---|---|
--wait | Wait for crawl to complete before returning |
--progress | Show progress while waiting |
--limit <n> | Max pages to crawl |
--max-depth <n> | Max link depth to follow |
--include-paths <paths> | Only crawl URLs matching these paths |
--exclude-paths <paths> | Skip URLs matching these paths |
--delay <ms> | Delay between requests |
--max-concurrency <n> | Max parallel crawl workers |
--pretty | Pretty print JSON output |
-o, --output <path> | Output file path |
Tips
- Always use
--waitwhen you need the results immediately. Without it, crawl returns a job ID for async polling. - Use
--include-pathsto scope the crawl — don't crawl an entire site when you only need one section. - Crawl consumes credits per page. Check
firecrawl credit-usagebefore large crawls.
See also
- firecrawl-scrape — scrape individual pages
- firecrawl-map — discover URLs before deciding to crawl
- firecrawl-download — download site to local files (uses map + scrape)
Related skills
Lark SharedWire Feishu/Lark Open Platform auth, tokens, and shared API helpers into agent-built bots and internal tools.514k
Runcomfy CliLet their coding agent call any RunComfy model from the terminal or scripts without managing separate SDKs.298k30
Skills CliDiscover, install, update, and manage reusable agent capabilities from the open Skillselion ecosystem directly inside their coding agent.270k71TzstCreate, extract, list, and script `.tzst` / `.tar.zst` archives from the terminal with the right compression, streaming, and conflict flags.267k71XdropPull files from an Xdrop share URL, decrypt them locally, and land outputs in a chosen directory with optional JSON for automation.266k71
WizardAuthor interactive bash setup wizards that open URLs, capture secrets, write .env upserts, and set GitHub Actions secrets step by step.104k202kHow it compares
Pick firecrawl-crawl for multi-page same-site extraction; pick single-page scrape skills for one URL.
FAQ
When should developers use firecrawl-crawl?
firecrawl-crawl applies when content is needed from many pages on one site—such as all /docs/ pages—using Firecrawl CLI crawl with depth limits, path filters, and concurrent extraction.
Which shell commands does firecrawl-crawl allow?
firecrawl-crawl permits Bash(firecrawl *) and Bash(npx firecrawl *) so agents can run Firecrawl crawl jobs from installed CLI or npx without a separate integration layer.
Is Firecrawl Crawl safe to install?
skills.sh reports 2 of 3 security scanners passed. Review the Security Audits panel on this page before installing in production.