
Agent Fetch
- 281 installs
- 301 repo stars
- Updated March 15, 2026
- teng-lin/agent-fetch
agent-fetch is a local CLI agent skill that fetches full article text from URLs using seven extraction strategies, browser impersonation, and 200–700ms typical latency instead of truncated summaries.
About
agent-fetch from teng-lin/agent-fetch is both a skill and an npx CLI that returns complete article markdown with headings, links, and lists rather than built-in fetch summaries. It runs seven extraction strategies with browser impersonation presets such as chrome-143, ios-safari-18, and android-chrome-143, supports Netscape cookie files, custom CSS selectors, multi-page crawl with default depth 3 and up to 100 pages, and local PDF text extraction. Commands include standard JSON fetch, --raw HTML, -q quiet markdown, --text plain output, crawl with include and exclude globs, and --json JSONL for batch processing. Typical fetches complete in 200–700ms with suggestedAction hints for retry_with_extract, wait_and_retry, or skip on failures. Developers invoke agent-fetch whenever agents need full page text for research, RAG prep, or API documentation ingestion.
- HTTP and API fetch patterns for agents
- Authentication and header handling
- Response parsing and normalization
- External data source integration
- Reusable agent tool-chain utilities
Agent Fetch by the numbers
- 281 all-time installs (skills.sh)
- +1 installs in the week ending Jul 27, 2026 (Skillselion tracking)
- Ranked #2,348 of 16,546 AI & Agent Building skills by installs in the Skillselion catalog
- Data as of Jul 27, 2026 (Skillselion catalog sync)
npx skills add https://github.com/teng-lin/agent-fetch --skill agent-fetchAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 281 |
|---|---|
| repo stars | ★ 301 |
| Last updated | March 15, 2026 |
| Repository | teng-lin/agent-fetch ↗ |
How do agents fetch full article text from URLs?
Give coding agents reliable HTTP fetch, auth, and response-parsing patterns when pulling APIs, web pages, or files into tool chains, RAG prep, or live data lookups.
Who is it for?
Developers and coding agents that need complete webpage or PDF text for research, RAG ingestion, or documentation parsing beyond truncated fetch summaries.
Skip if: Skip agent-fetch when you need authenticated interactive browser automation or JavaScript-heavy SPAs that require a full headless browser session.
When should I use this skill?
User asks to read a URL, fetch article content, crawl documentation sites, extract PDF text, or built-in web fetch returned incomplete pages.
What you get
Structured markdown or JSON article output with title, byline, source, latencyMs, and full text content preserved.
- Structured article markdown
- JSON fetch metadata
- Crawl JSONL output
By the numbers
- Seven content extraction strategies documented
- Typical fetch latency range 200–700ms in skill description
- Crawl defaults: depth 3 and max 100 pages
Files
agent-fetch Skill
A better web fetch for text content. Your built-in web fetch summarizes or truncates pages. agent-fetch extracts the complete article — every paragraph, heading, and link — using 7 extraction strategies and browser impersonation. No server required, runs as a local CLI tool.
When to Use This Skill
Use agent-fetch whenever you need to read a URL. It returns full article text with structure preserved — better than your built-in web fetch for any task involving page content.
- User asks to read, fetch, or analyze a URL
- User types
/agent-fetch <url> - You need the full text, not a summary or truncation
- Your built-in web fetch returned incomplete or garbled content
Prerequisites
agent-fetch runs via npx (no install needed):
npx agent-fetch --helpCommands
/agent-fetch <url> - Fetch and Extract Article
Default usage. Fetches URL with browser impersonation and extracts complete article content as markdown.
npx agent-fetch "<url>" --jsonParse the JSON output and present to the user:
---
title: {title}
author: {byline || "Unknown"}
source: {siteName}
url: {url}
date: {publishedTime || "Unknown"}
fetched_in: {latencyMs}ms
---
## {markdown || textContent}
{markdown || textContent}If fetch fails, check suggestedAction in the JSON:
| suggestedAction | What it means | Next action |
|---|---|---|
retry_with_extract | Needs full browser | Inform user; agent-fetch is HTTP-only |
wait_and_retry | Rate limited | Wait 60s and retry |
skip | Cannot access this site | Inform user |
/agent-fetch raw <url> - Raw HTML
Fetch raw HTML without extraction.
npx agent-fetch "<url>" --raw/agent-fetch quiet <url> - Markdown Only
Just the article markdown, no metadata.
npx agent-fetch "<url>" -q/agent-fetch text <url> - Plain Text Only
Plain text content without formatting or metadata.
npx agent-fetch "<url>" --text/agent-fetch cookies - Use Persistent Cookies
Load cookies from a Netscape format file or pass inline:
# From Netscape cookie file (export from browser)
npx agent-fetch "<url>" --cookie-file ~/.cookies.txt
# Inline cookies (repeatable)
npx agent-fetch "<url>" --cookie "sessionId=abc123; theme=dark"/agent-fetch selectors <url> - Custom CSS Selectors
Extract specific elements or remove unwanted ones:
# Extract only the article, remove navigation and ads
npx agent-fetch "<url>" --select "article" --remove "nav, .sidebar, [class*='ad']"
# Extract all divs with class "post-content"
npx agent-fetch "<url>" --select ".post-content"/agent-fetch crawl <url> - Crawl Multiple Pages
Follow links and extract content from multiple pages:
# Crawl with defaults (depth: 3, max 100 pages)
npx agent-fetch crawl "<url>"
# Deeper crawl with concurrency control
npx agent-fetch crawl "<url>" --depth 5 --limit 50 --concurrency 3
# Include/exclude specific URL patterns
npx agent-fetch crawl "<url>" --include "*/blog/*" --exclude "**/archive/**"
# Add rate limiting delay between requests
npx agent-fetch crawl "<url>" --delay 1000
# Allow cross-origin (stay on same origin by default)
npx agent-fetch crawl "<url>" --no-same-origin
# Output as JSONL for processing
npx agent-fetch crawl "<url>" --json/agent-fetch pdf <file> - Extract from PDF
Extract text content from local PDF files:
# Extract PDF as markdown with metadata
npx agent-fetch document.pdf
# JSON output for programmatic access
npx agent-fetch document.pdf --json
# Just the text content
npx agent-fetch document.pdf --text/agent-fetch preset - Custom TLS Fingerprint
Impersonate different browsers to bypass fingerprinting checks:
# Chrome 143 (default)
npx agent-fetch "<url>" --preset "chrome-143"
# iOS Safari 18
npx agent-fetch "<url>" --preset "ios-safari-18"
# Android Chrome 143
npx agent-fetch "<url>" --preset "android-chrome-143"Related skills
How it compares
Use agent-fetch for fast full-text HTTP extraction in agent pipelines; use browser automation MCP tools when sites require full JavaScript rendering sessions.
FAQ
How do you run agent-fetch from an agent?
agent-fetch executes via npx agent-fetch "<url>" --json with no install required. The skill parses JSON fields such as title, markdown, textContent, latencyMs, and suggestedAction to present full article content to the user.
What extraction options does agent-fetch support?
agent-fetch offers seven extraction strategies, browser presets like chrome-143 and ios-safari-18, Netscape cookie files, --select and --remove CSS selectors, multi-page crawl with depth and concurrency flags, and local PDF text extraction.
How fast is agent-fetch compared to built-in fetch?
agent-fetch skill metadata cites 200–700ms typical latency while returning complete structured article text. Built-in agent fetch often summarizes or truncates pages, which agent-fetch avoids by design.