Web Fetch

Web Fetch Skill

This skill enables Claude to safely retrieve and analyze web content from user-provided URLs or search results while implementing security controls to prevent data exfiltration.

When to Use This Skill

Use this skill when:

• User provides a URL to fetch and analyze
• User asks to retrieve content from a web page
• User wants to follow up on WebSearch results by fetching full content
• User needs to extract information from a PDF hosted online
• User requests analysis of documentation, blog posts, or web resources

Security Controls

Domain Filtering

The WebFetch tool supports domain filtering to restrict which sites can be accessed:

1. Allowed Domains - Whitelist approach (only these domains permitted) 2. Blocked Domains - Blacklist approach (all except these domains)

IMPORTANT: Never build or guess URLs. Only use:

• URLs explicitly provided by the user
• URLs returned from WebSearch results
• URLs found in local files (with user confirmation)

Max Uses Tracking

To prevent data exfiltration, track the number of WebFetch calls per session:

• Default limit: 10 fetches per conversation (adjustable based on user needs)
• Warning threshold: Alert user at 7 fetches
• Hard stop: Require explicit user approval beyond limit

Usage Guidelines

1. Single URL Fetch

When user provides a single URL:

User: "Can you fetch and analyze https://example.com/article"

Steps:
1. Validate the URL is user-provided (not generated)
2. Determine appropriate prompt for analysis
3. Use WebFetch with clear, specific prompt
4. Increment fetch counter
5. Analyze and present results

Example:

WebFetch:
  url: https://example.com/article
  prompt: Extract and summarize the main points of this article, including key arguments and conclusions.

2. Multiple URLs from Search Results

When following up on WebSearch results:

User: "Search for Python async best practices and analyze the top results"

Steps:
1. Perform WebSearch first
2. Present search results to user
3. Ask user which URLs to fetch (don't assume)
4. Fetch user-approved URLs sequentially
5. Track total fetches against max_uses
6. Synthesize findings across sources

3. Domain Filtering

When user specifies domain restrictions:

User: "Fetch Python docs but avoid third-party blogs"

Apply filters:
- allowed_domains: ["docs.python.org", "peps.python.org"]
OR
- blocked_domains: ["medium.com", "dev.to", "blogger.com"]

Example:

WebFetch:
  url: https://docs.python.org/3/library/asyncio.html
  prompt: Explain the asyncio event loop and provide key usage examples
  allowed_domains: ["docs.python.org", "peps.python.org"]

4. PDF Content Retrieval

When fetching PDF documents:

User: "Can you analyze this research paper: https://example.com/paper.pdf"

Steps:
1. Verify URL ends in .pdf or user confirms it's a PDF
2. Use specific prompt for academic content
3. WebFetch automatically processes PDF pages
4. Extract text and visual content
5. Provide structured analysis

Example:

WebFetch:
  url: https://example.com/paper.pdf
  prompt: Extract the abstract, methodology, key findings, and conclusions from this research paper.

Workflow Pattern

1. Receive URL request from user
   ↓
2. Validate URL source (user-provided/search result)
   ↓
3. Check fetch counter against max_uses
   ↓
4. Apply domain filters if specified
   ↓
5. Construct specific, clear prompt
   ↓
6. Execute WebFetch tool
   ↓
7. Increment counter
   ↓
8. Analyze results
   ↓
9. Present findings to user

Max Uses Implementation

Track fetches within conversation:

Fetch Count Tracking:
- Initialize: fetch_count = 0
- After each WebFetch: fetch_count += 1
- At 7 fetches: "Note: Approaching fetch limit (7/10). Let me know if you need to adjust."
- At 10 fetches: "Reached fetch limit (10/10). Do you want to continue? This helps prevent unintended data access."
- Beyond 10: Require explicit user "yes" before proceeding

Reset Conditions

The fetch counter resets:

• At the start of each new conversation
• When user explicitly requests reset
• Never automatically mid-conversation

Prompt Crafting Best Practices

Create specific, actionable prompts for WebFetch:

Good Prompts ✓

• "Extract the installation instructions and list all dependencies mentioned"
• "Summarize the main argument and supporting evidence from this article"
• "List all API endpoints documented on this page with their parameters"
• "Extract code examples showing async/await usage"

Poor Prompts ✗

• "Tell me about this page" (too vague)
• "What does this say?" (unclear goal)
• "Read this" (no analysis requested)
• "Everything" (overly broad)

Error Handling

Redirect Detection

When WebFetch returns a redirect message:

1. WebFetch indicates redirect to different host
2. Present redirect URL to user
3. Ask for confirmation to follow
4. Make new WebFetch with redirect URL
5. Count as separate fetch

Fetch Failures

If WebFetch fails:

1. Check URL format (valid, fully-formed)
2. Verify domain filters aren't blocking
3. Confirm URL is accessible (not behind auth)
4. Suggest alternative approach (search for cached version)
5. Don't retry automatically (counts against max_uses)

Examples

Example 1: Simple Article Fetch

User: "Can you fetch and summarize https://example.com/blog/new-features"

Response:
1. Use WebFetch with:
   - url: https://example.com/blog/new-features
   - prompt: "Summarize the new features described, including their benefits and any code examples provided"
2. Fetch count: 1/10
3. Present summary with key points

Example 2: Search + Selective Fetch

User: "Find articles about Rust error handling and analyze the best one"

Response:
1. WebSearch: "Rust error handling best practices"
2. Present top 5 results
3. User selects: "The second one looks good"
4. WebFetch selected URL only
5. Fetch count: 1/10
6. Analyze and explain Rust error handling patterns

Example 3: Multiple Fetches with Domain Filter

User: "Compare official Python and Rust async documentation"

Response:
1. WebFetch: https://docs.python.org/3/library/asyncio.html
   - allowed_domains: ["docs.python.org"]
   - prompt: "Explain Python's async/await model and event loop"
   - Fetch count: 1/10

2. WebFetch: https://doc.rust-lang.org/book/async-await.html
   - allowed_domains: ["doc.rust-lang.org"]
   - prompt: "Explain Rust's async/await model and futures"
   - Fetch count: 2/10

3. Synthesize comparison

Example 4: PDF Research Paper

User: "Analyze this ML paper: https://arxiv.org/pdf/2301.12345.pdf"

Response:
1. WebFetch with:
   - url: https://arxiv.org/pdf/2301.12345.pdf
   - prompt: "Extract abstract, methodology, datasets used, key results, and main conclusions from this machine learning research paper"
2. Fetch count: 1/10
3. Present structured analysis:
   - Abstract summary
   - Methods overview
   - Key findings
   - Conclusions and implications

Security Checklist

Before each WebFetch, verify:

• [ ] URL is user-provided or from search results (not generated)
• [ ] Fetch count is within limits or user approved
• [ ] Domain filters applied if specified
• [ ] Prompt is specific and purposeful
• [ ] Not fetching sensitive/internal URLs
• [ ] User understands what will be fetched

Integration with Other Skills

This skill works well with:

• summarization: Fetch content, then summarize it
• code-review: Fetch code examples for analysis
• research: Gather information from multiple sources
• learning: Fetch documentation and tutorials

Limitations

Be aware of:

1. Cannot build URLs: Only use explicitly provided URLs 2. No authentication: Cannot fetch content behind login 3. Rate limits: Some sites may block automated requests 4. Large content: Very large pages may be summarized 5. Dynamic content: JavaScript-rendered content may not be available 6. Cache timing: 15-minute cache may show stale content

Best Practices Summary

1. ✅ Always use user-provided or search-returned URLs 2. ✅ Track fetch count and warn before limits 3. ✅ Write specific, actionable prompts 4. ✅ Apply domain filters when appropriate 5. ✅ Handle redirects explicitly 6. ✅ Present findings clearly to user 7. ❌ Never generate or guess URLs 8. ❌ Never auto-fetch without user intent 9. ❌ Never bypass max_uses without approval 10. ❌ Never fetch sensitive/internal domains

Quick Reference

---

Remember: This skill prioritizes security through URL validation, domain filtering, and usage limits while enabling powerful web content retrieval capabilities.

Web Fetch Skill - Examples

This document provides practical, real-world examples of using the web-fetch skill.

Example 1: Fetching Documentation

Scenario

User wants to understand Python's asyncio library.

Conversation

User: Can you fetch the Python asyncio documentation and explain the event loop?

Claude:
I'll fetch the official Python asyncio documentation to explain the event loop.

[Executes:]
WebFetch(
    url="https://docs.python.org/3/library/asyncio-eventloop.html",
    prompt="Explain what an event loop is, how it works, and provide key usage patterns with code examples"
)

[Fetch count: 1/10]

[After receiving results:]
The asyncio event loop is the core of Python's asynchronous programming model. Here's what I learned from the official documentation:

**What is an Event Loop?**
The event loop is the central execution device that manages and distributes the execution of async tasks and callbacks. It runs in a single thread and coordinates all asynchronous operations.

**How it Works:**
1. The event loop continuously checks for tasks ready to run
2. Executes ready tasks until they hit an `await` point
3. Switches to other ready tasks while waiting
4. Handles I/O events and callbacks
5. Repeats until all tasks complete

**Key Usage Pattern:**

import asyncio

async def main(): await asyncio.sleep(1) print('Done')

Run the event loop

asyncio.run(main())

Would you like me to fetch more specific information about any particular aspect of asyncio?

Key Points

• ✅ Used user-provided URL (implicit - official docs)
• ✅ Specific prompt targeting the user's question
• ✅ Tracked fetch count (1/10)
• ✅ Presented clear, structured results
• ✅ Offered follow-up options

---

Example 2: Research with WebSearch Integration

Scenario

User wants to research Rust error handling best practices.

Conversation

User: Find and analyze the best articles about Rust error handling.

Claude:
I'll search for articles on Rust error handling and then fetch the most relevant ones.

[Executes:]
WebSearch(query="Rust error handling best practices 2024")

[After receiving search results:]
I found several excellent resources. Here are the top results:

1. "Error Handling in Rust" - Official Rust Book (doc.rust-lang.org)
2. "Rust Error Handling Patterns" - blog.burntsushi.net
3. "The ? Operator and Error Propagation" - rust-lang.org
4. "Custom Error Types in Rust" - fettblog.eu
5. "anyhow vs thiserror: Which to Use?" - dev.to

I recommend fetching #1 (Official Rust Book) and #2 (Andrew Gallant's blog) for comprehensive coverage. Should I proceed?

User: Yes, fetch both.

Claude:
I'll fetch both articles now.

[Executes:]
WebFetch(
    url="https://doc.rust-lang.org/book/ch09-00-error-handling.html",
    prompt="Extract and explain: 1) Result and Option types, 2) the ? operator, 3) best practices for error handling, 4) code examples",
    allowed_domains=["doc.rust-lang.org", "rust-lang.org"]
)
[Fetch count: 1/10]

WebFetch(
    url="https://blog.burntsushi.net/rust-error-handling",
    prompt="Extract: 1) error handling patterns, 2) library recommendations (anyhow, thiserror), 3) real-world examples, 4) common pitfalls",
    allowed_domains=["blog.burntsushi.net"]
)
[Fetch count: 2/10]

[After receiving both results:]
I've analyzed both resources. Here's a comprehensive guide to Rust error handling:

## Result and Option Types

**From Official Rust Book:**
- `Result<T, E>` for operations that can fail
- `Option<T>` for values that may be absent
- Pattern matching for explicit handling

**Code example:**

fn read_file(path: &str) -> Result<String, io::Error> { fs::read_to_string(path) }

match read_file("data.txt") { Ok(contents) => println!("{}", contents), Err(e) => eprintln!("Error: {}", e), }

## The ? Operator

**From both sources:**
The `?` operator propagates errors up the call stack:

fn process() -> Result<(), Box<dyn Error>> { let data = read_file("data.txt")?; // Returns error if fails let parsed = parse_data(&data)?; // Chains error handling Ok(()) }

## Library Recommendations

**From Andrew Gallant's blog:**

1. **anyhow**: For applications (flexible error handling)

use anyhow::{Context, Result};

fn app_logic() -> Result<()> { let config = load_config() .context("Failed to load config")?; Ok(()) }

2. **thiserror**: For libraries (structured error types)

use thiserror::Error;

#[derive(Error, Debug)] enum MyError { #[error("IO error: {0}")] Io(#[from] io::Error),

#[error("Parse error: {0}")] Parse(String), }

## Best Practices

1. Use `Result` for recoverable errors
2. Use `panic!` only for bugs/invariants
3. Provide context with error messages
4. Use `?` operator for clean error propagation
5. In libraries: define custom error types
6. In applications: use `anyhow` for convenience

Would you like me to fetch more specific information about any of these topics?

Key Points

• ✅ Combined WebSearch + WebFetch workflow
• ✅ Asked user which results to fetch (didn't assume)
• ✅ Applied domain filtering for security
• ✅ Fetched sequentially with tracking (2/10)
• ✅ Synthesized findings from multiple sources
• ✅ Presented comprehensive, structured analysis

---

Example 3: PDF Research Paper Analysis

Scenario

User wants to analyze a machine learning research paper.

Conversation

User: Can you analyze this paper and summarize the key contributions? https://arxiv.org/pdf/2103.14030.pdf

Claude:
I'll fetch and analyze this ML research paper from arXiv.

[Executes:]
WebFetch(
    url="https://arxiv.org/pdf/2103.14030.pdf",
    prompt="Extract and summarize: 1) Paper title and authors, 2) Abstract/main contribution, 3) Methodology and approach, 4) Key experimental results, 5) Conclusions and future work, 6) Top 3 cited references"
)
[Fetch count: 1/10]

[After receiving PDF content:]
# Paper Analysis: [Paper Title]

## Title and Authors
**Title:** "Learning Transferable Visual Models From Natural Language Supervision"
**Authors:** Alec Radford, et al. (OpenAI)
**Published:** 2021

## Main Contribution
This paper introduces CLIP (Contrastive Language-Image Pre-training), a neural network that learns visual concepts from natural language supervision. Key innovation: training on 400 million (image, text) pairs from the internet rather than fixed classification labels.

## Methodology
**Approach:**
1. Contrastive pre-training on image-text pairs
2. Joint embedding space for images and text
3. Zero-shot transfer to downstream tasks
4. No task-specific fine-tuning required

**Architecture:**
- Image encoder: Vision Transformer (ViT) or ResNet
- Text encoder: Transformer
- Contrastive loss function aligns embeddings

## Key Results
- Zero-shot CLIP matches supervised ResNet-50 on ImageNet
- Robust to distribution shift (superior generalization)
- Competitive with task-specific models across 30+ datasets
- Efficient: learns from web-scale noisy data

## Conclusions
1. Natural language supervision enables flexible, generalizable visual learning
2. Zero-shot transfer competitive with supervised approaches
3. Scalability: performance improves with model and data size
4. Opens path for more general-purpose vision systems

## Top Cited References
1. "Attention Is All You Need" (Vaswani et al.) - Transformer architecture
2. "An Image is Worth 16x16 Words" (Dosovitskiy et al.) - Vision Transformer
3. "SimCLR" (Chen et al.) - Contrastive learning framework

**Analysis:**
This is a foundational paper in multi-modal learning. CLIP's key insight is using natural language as a supervision signal, which provides much richer information than traditional labels. The zero-shot capability is particularly impressive - the model can classify images into arbitrary categories it wasn't explicitly trained on.

Would you like me to:
1. Fetch any of the cited references for deeper understanding?
2. Search for follow-up papers that build on CLIP?
3. Look for code implementations or tutorials?

Key Points

• ✅ PDF URL directly from user
• ✅ Structured prompt for academic content
• ✅ Comprehensive extraction (abstract, methods, results, citations)
• ✅ Clear presentation with sections
• ✅ Offered relevant follow-up actions

---

Example 4: Approaching Fetch Limit

Scenario

User conducting extensive research, approaching the fetch limit.

Conversation

User: I need to compare authentication methods across 10 different frameworks.

Claude:
I understand you want to compare authentication approaches across 10 frameworks. Before we start, I should mention that the default fetch limit is 10 per conversation to prevent unintended data access.

For your task, I'll need to fetch documentation for each framework. Let me proceed efficiently:

1. Which 10 frameworks would you like me to analyze?
2. Should I focus on specific aspects (OAuth, JWT, sessions, etc.)?

User: Django, Flask, Express, FastAPI, Spring Boot, Laravel, Rails, ASP.NET, Phoenix, Gin. Focus on JWT and OAuth support.

Claude:
Perfect. I'll fetch JWT and OAuth documentation for each framework. Let me start:

[Executes for each framework:]
1. Django: WebFetch(...) [Fetch: 1/10]
2. Flask: WebFetch(...) [Fetch: 2/10]
3. Express: WebFetch(...) [Fetch: 3/10]
4. FastAPI: WebFetch(...) [Fetch: 4/10]
5. Spring Boot: WebFetch(...) [Fetch: 5/10]
6. Laravel: WebFetch(...) [Fetch: 6/10]
7. Rails: WebFetch(...) [Fetch: 7/10]

⚠️ **Note:** Approaching fetch limit (7/10). Continuing with remaining 3...

8. ASP.NET: WebFetch(...) [Fetch: 8/10]
9. Phoenix: WebFetch(...) [Fetch: 9/10]
10. Gin: WebFetch(...) [Fetch: 10/10]

✅ **Fetch limit reached (10/10)** - All frameworks analyzed.

[After collecting all data:]
# Authentication Comparison: 10 Web Frameworks

## JWT Support Summary

| Framework | Native Support | Library | Ease of Use |
|-----------|----------------|---------|-------------|
| Django | Via package | djangorestframework-simplejwt | ⭐⭐⭐⭐ |
| Flask | Via extension | Flask-JWT-Extended | ⭐⭐⭐⭐⭐ |
| Express | Via middleware | jsonwebtoken | ⭐⭐⭐⭐⭐ |
| FastAPI | Via dependency | python-jose | ⭐⭐⭐⭐⭐ |
| Spring Boot | Built-in | Spring Security + JWT | ⭐⭐⭐ |
| Laravel | Via package | tymon/jwt-auth | ⭐⭐⭐⭐ |
| Rails | Via gem | jwt | ⭐⭐⭐⭐ |
| ASP.NET | Built-in | JwtBearer | ⭐⭐⭐ |
| Phoenix | Via library | Guardian | ⭐⭐⭐⭐ |
| Gin | Via middleware | jwt-go | ⭐⭐⭐⭐⭐ |

## OAuth Support Summary

| Framework | OAuth 2.0 | Social Login | Provider Library |
|-----------|-----------|--------------|------------------|
| Django | ✅ | ✅ | django-allauth |
| Flask | ✅ | ✅ | Authlib |
| Express | ✅ | ✅ | Passport.js |
| FastAPI | ✅ | ✅ | Authlib |
| Spring Boot | ✅ | ✅ | Spring Security OAuth |
| Laravel | ✅ | ✅ | Socialite |
| Rails | ✅ | ✅ | OmniAuth |
| ASP.NET | ✅ | ✅ | IdentityServer |
| Phoenix | ✅ | ✅ | Ueberauth |
| Gin | ✅ | ✅ | goth |

## Recommendations by Use Case

**Fastest Setup (JWT):**
1. FastAPI - Excellent docs, minimal boilerplate
2. Express - Simple middleware pattern
3. Flask - Flask-JWT-Extended is very intuitive

**Best Enterprise OAuth:**
1. Spring Boot - Comprehensive, battle-tested
2. ASP.NET - Native Microsoft integration
3. Django - Mature ecosystem

**Detailed Implementation Guides:**
[Presents framework-specific implementation patterns extracted from each fetch]

---

**Note:** We've reached the fetch limit (10/10) for this conversation. If you need additional information, we can:
1. Start a new conversation to reset the counter
2. I can analyze based on what I've already fetched
3. You can adjust the limit if needed for research tasks

What would you like to do next?

Key Points

• ✅ Warned user about fetch limit upfront
• ✅ Efficient fetching (one per framework)
• ✅ Clear progress tracking (X/10 after each fetch)
• ✅ Warning at 7/10 threshold
• ✅ Notification when limit reached
• ✅ Comprehensive synthesis of all fetched data
• ✅ Explained options when limit reached

---

Example 5: Domain Filtering for Security

Scenario

Corporate user wants to fetch only from approved internal documentation.

Conversation

User: I need to fetch our company's deployment guides. Only fetch from wiki.company.com and docs.company.com for security reasons.

Claude:
Understood. I'll restrict fetches to only your approved internal domains: wiki.company.com and docs.company.com.

What specific deployment guide would you like me to fetch?

User: Start with https://wiki.company.com/engineering/kubernetes-deploy

Claude:
I'll fetch the Kubernetes deployment guide from your internal wiki with domain restrictions applied.

[Executes:]
WebFetch(
    url="https://wiki.company.com/engineering/kubernetes-deploy",
    prompt="Extract: 1) Prerequisites, 2) Step-by-step deployment process, 3) Configuration requirements, 4) Common troubleshooting steps",
    allowed_domains=["wiki.company.com", "docs.company.com"]
)
[Fetch count: 1/10]

[After receiving content:]
# Kubernetes Deployment Guide (Internal)

## Prerequisites
- kubectl v1.28+ installed
- Access to company-k8s cluster
- ServiceAccount with deployment permissions
- Docker images in company registry

## Deployment Process

**Step 1: Configure kubectl context**

kubectl config use-context company-prod kubectl get nodes # Verify access

**Step 2: Apply manifests**

kubectl apply -f deployments/ kubectl apply -f services/ kubectl apply -f ingress/

**Step 3: Verify deployment**

kubectl get pods -n your-namespace kubectl logs deployment/your-app

## Configuration Requirements
[Details from internal guide...]

## Troubleshooting
[Common issues and solutions...]

The guide also references https://docs.company.com/k8s-troubleshooting. Would you like me to fetch that as well? (Domain is approved)

User: Yes, fetch the troubleshooting guide.

Claude:
[Executes:]
WebFetch(
    url="https://docs.company.com/k8s-troubleshooting",
    prompt="Extract all troubleshooting scenarios, error messages, and resolution steps",
    allowed_domains=["wiki.company.com", "docs.company.com"]
)
[Fetch count: 2/10]

[After receiving content:]
# Kubernetes Troubleshooting Guide (Internal)

## Common Issues
[Detailed troubleshooting from internal docs...]

---

Both guides fetched successfully with domain restrictions active. All fetches limited to:
✅ wiki.company.com
✅ docs.company.com

If I encounter any links to external domains, I'll notify you before attempting to fetch them.

Key Points

• ✅ Applied allowed_domains filter from the start
• ✅ Only fetched from approved corporate domains
• ✅ Asked before fetching additional URLs
• ✅ Verified domain compliance explicitly
• ✅ Would reject external domains automatically

---

Example 6: Handling Redirects

Scenario

User provides URL that redirects to different domain.

Conversation

User: Fetch this article: https://bit.ly/rust-async-guide

Claude:
I'll fetch the article from that URL.

[Executes:]
WebFetch(
    url="https://bit.ly/rust-async-guide",
    prompt="Explain Rust async programming concepts and patterns"
)

[WebFetch returns redirect message:]
"Redirect detected: https://bit.ly/rust-async-guide → https://tokio.rs/tokio/tutorial

Claude:
The short URL redirects to https://tokio.rs/tokio/tutorial (the official Tokio tutorial). Should I fetch the content from the redirect destination?

User: Yes, go ahead.

Claude:
[Executes:]
WebFetch(
    url="https://tokio.rs/tokio/tutorial",
    prompt="Extract and explain: 1) Introduction to Tokio, 2) Core concepts (runtime, tasks, async/await), 3) Key examples and patterns",
    allowed_domains=["tokio.rs"]
)
[Fetch count: 2/10] (both redirect and destination counted)

[After receiving content:]
# Tokio Tutorial Summary

## Introduction
Tokio is an asynchronous runtime for Rust that provides:
- Multi-threaded, work-stealing task scheduler
- Async I/O backed by the OS event queue (epoll, kqueue, IOCP)
- Async TCP/UDP sockets, timers, and channels

## Core Concepts

**1. Runtime:**
The Tokio runtime executes async code:

#[tokio::main] async fn main() { println!("Hello from Tokio!"); }

**2. Tasks:**
Lightweight threads of execution:

tokio::spawn(async { // Runs concurrently });

**3. Async/Await:**
[Details from tutorial...]

---

Note: Followed redirect from bit.ly → tokio.rs. Total fetches: 2/10 (initial + redirect).

Key Points

• ✅ Detected redirect automatically
• ✅ Informed user about destination
• ✅ Asked for confirmation before following
• ✅ Counted both as separate fetches
• ✅ Applied domain filter to final destination
• ✅ Transparent about redirect handling

---

Example 7: Efficient Multi-Source Research

Scenario

User wants to learn a new concept from multiple perspectives.

Conversation

User: I want to understand Docker multi-stage builds. Find and compare explanations from official docs and community tutorials.

Claude:
I'll help you understand Docker multi-stage builds by fetching both official documentation and a highly-rated community tutorial.

**Plan:**
1. Fetch official Docker docs on multi-stage builds
2. Fetch a well-regarded community tutorial
3. Synthesize both perspectives

Let me start:

[Executes:]
WebFetch(
    url="https://docs.docker.com/build/building/multi-stage/",
    prompt="Extract: 1) What multi-stage builds are, 2) Why they're useful, 3) Syntax and structure, 4) Complete example, 5) Best practices",
    allowed_domains=["docs.docker.com"]
)
[Fetch count: 1/10]

[After first fetch:]
**From Official Docs:**
Multi-stage builds use multiple FROM statements in a Dockerfile, allowing you to:
- Copy artifacts between stages
- Reduce final image size
- Separate build-time and runtime dependencies

Example:

Stage 1: Build

FROM golang:1.21 AS builder WORKDIR /app COPY . . RUN go build -o main .

Stage 2: Runtime

FROM alpine:latest COPY --from=builder /app/main /main CMD ["/main"]

Now fetching community perspective:

[Executes:]
WebSearch("Docker multi-stage builds tutorial")
[Finds highly-rated tutorial on dev.to]

WebFetch(
    url="https://dev.to/author/docker-multistage-guide",
    prompt="Extract: 1) Real-world use cases, 2) Practical examples, 3) Common pitfalls, 4) Performance tips, 5) Comparison with single-stage"
)
[Fetch count: 2/10]

[After second fetch:]
**From Community Tutorial:**
Real-world benefits:
- Image size: 1.2GB → 15MB (Python app example)
- Security: No build tools in production image
- Speed: Cached layers optimize rebuild time

Common pitfalls:
- Forgetting to name stages (use AS)
- Copying unnecessary files between stages
- Not leveraging BuildKit features

# Synthesized Understanding

## What Are Multi-Stage Builds?
[Combined explanation from both sources]

## Why Use Them?
**Official perspective:** Separate concerns, reduce size
**Community perspective:** Real-world 80x size reduction examples

## Complete Example (Best Practices from Both Sources)

Stage 1: Dependencies

FROM node:18 AS deps WORKDIR /app COPY package*.json ./ RUN npm ci --only=production

Stage 2: Build

FROM node:18 AS builder WORKDIR /app COPY package*.json ./ RUN npm ci COPY . . RUN npm run build

Stage 3: Production

FROM node:18-alpine WORKDIR /app COPY --from=deps /app/node_modules ./node_modules COPY --from=builder /app/dist ./dist CMD ["node", "dist/index.js"]

## Key Insights

**From Official Docs:**
- Use numbered stages or names
- COPY --from=<stage> to transfer artifacts
- Each FROM starts fresh

**From Community:**
- Alpine for minimal runtime images
- Cache dependencies separately
- Security: fewer attack vectors

**Comparison Table:**
| Aspect | Single-Stage | Multi-Stage |
|--------|--------------|-------------|
| Image Size | ~1GB | ~15MB |
| Build Tools in Prod | Yes ❌ | No ✅ |
| Security Surface | Large | Minimal |
| Build Time | Fast | Slightly slower |

## When to Use Multi-Stage Builds
1. Compiled languages (Go, Rust, C++)
2. Frontend builds (npm run build)
3. Any app with build-time vs runtime dependencies
4. Production deployments (size matters)

This synthesis combines the official technical specification with real-world practical insights. The community tutorial's before/after examples really demonstrate the dramatic size reduction possible.

Fetches used: 2/10. Would you like me to:
1. Find Dockerfile examples for specific languages?
2. Fetch information about advanced multi-stage patterns?
3. Look up BuildKit-specific optimizations?

Key Points

• ✅ Strategic fetching (official + community)
• ✅ Combined WebSearch + WebFetch
• ✅ Synthesized multiple perspectives
• ✅ Presented comparison table
• ✅ Provided complete practical examples
• ✅ Tracked fetches (2/10)
• ✅ Offered relevant follow-ups

---

Example 8: Fetch Limit Exceeded - Handling

Scenario

User conducting very extensive research exceeds the limit.

Conversation

User: I need to analyze 15 different CI/CD platforms. Fetch documentation for each.

Claude:
I understand you want to analyze 15 CI/CD platforms. However, the fetch limit is 10 per conversation to prevent unintended data access.

**Options:**
1. **Prioritize**: Select the top 10 platforms to analyze now
2. **Two-phase**: Fetch 10 now, then continue in new conversation for remaining 5
3. **Increase limit**: If this is an authorized research task, we can proceed with user approval beyond 10

Which approach works best for you?

User: Let's do the top 10 first. GitHub Actions, GitLab CI, Jenkins, CircleCI, Travis CI, Azure Pipelines, Bitbucket Pipelines, Drone, TeamCity, Bamboo.

Claude:
Perfect. I'll fetch documentation for these 10 platforms focusing on key features, pricing, and integration capabilities.

[Proceeds to fetch 10 platforms...]
[After 10 fetches:] ✅ Fetch limit reached (10/10)

[Presents comprehensive comparison of all 10 platforms]

Now, for the remaining 5 platforms (Buildkite, Semaphore, Codefresh, ArgoCD, Tekton), we have two options:

1. **New Conversation**: Start fresh with reset counter
2. **Proceed with Approval**: I can continue fetching beyond the limit with your explicit approval

If you choose option 2, please confirm: "Yes, continue fetching beyond limit for research purposes."

User: Yes, continue fetching beyond limit for research purposes.

Claude:
Thank you for the approval. Continuing with the remaining 5 platforms...

⚠️ **Fetching beyond standard limit with user approval**

[Executes:]
WebFetch(url for Buildkite...)
[Fetch count: 11/15 - BEYOND LIMIT with approval]

[Executes:]
WebFetch(url for Semaphore...)
[Fetch count: 12/15 - BEYOND LIMIT with approval]

[Continues through all 15...]

✅ **All 15 platforms analyzed** (10 standard + 5 with approval)

[Presents comprehensive comparison of all 15 CI/CD platforms]

For future extensive research tasks, consider:
- Starting a new conversation after 10 fetches
- Using the search-first, selective-fetch strategy
- Requesting limit adjustment at the start if known large task

Key Points

• ✅ Warned user about limit upfront
• ✅ Offered multiple approaches
• ✅ Required explicit approval to exceed
• ✅ Clear notation when beyond limit
• ✅ Tracked all fetches transparently (11/15, 12/15, etc.)
• ✅ Completed comprehensive analysis
• ✅ Provided guidance for future similar tasks

---

Summary of Best Practices from Examples

1. URL Handling

• ✅ Always use user-provided or search-returned URLs
• ✅ Handle redirects explicitly with user notification
• ✅ Apply domain filtering when security matters
• ❌ Never generate or guess URLs

2. Fetch Tracking

• ✅ Track count after every fetch (X/10)
• ✅ Warn at threshold (7/10)
• ✅ Require approval beyond limit
• ✅ Be transparent about count status

3. Prompt Quality

• ✅ Specific, structured prompts (numbered points)
• ✅ Target user's actual question
• ✅ Request examples and practical guidance
• ❌ Avoid vague "tell me about" prompts

4. Workflow Efficiency

• ✅ Search first, fetch selectively
• ✅ Ask user which results to fetch
• ✅ Synthesize multiple sources
• ✅ Present structured, scannable results

5. User Communication

• ✅ Explain what you're fetching and why
• ✅ Show progress (fetch count)
• ✅ Offer relevant follow-ups
• ✅ Be transparent about limits and redirects

6. Security

• ✅ Apply domain filters for corporate/sensitive scenarios
• ✅ Never bypass filters without approval
• ✅ Track fetches to prevent exfiltration
• ✅ Respect user-defined security boundaries

---

Quick Reference: Example Scenarios

These examples demonstrate the web-fetch skill in realistic scenarios, showcasing proper URL handling, fetch tracking, domain filtering, and security-conscious usage patterns.

Web Fetch Skill

A Claude Code skill for safely retrieving and analyzing web content with built-in security controls.

Overview

The web-fetch skill enables Claude to retrieve content from user-provided URLs or search results while implementing security measures to prevent data exfiltration. It handles domain filtering, usage limits, and proper URL validation.

Features

• ✅ Safe URL Handling: Only uses user-provided or search-returned URLs (never generates URLs)
• ✅ Domain Filtering: Whitelist/blacklist capabilities for access control
• ✅ Usage Limits: Tracks fetches per conversation (default: 10) to prevent data exfiltration
• ✅ PDF Support: Automatically processes PDF documents
• ✅ Redirect Handling: Explicitly manages URL redirects with user awareness
• ✅ Structured Analysis: Guides Claude to extract and present information effectively

Installation

This skill is included in the .claude/skills/ directory and is automatically discovered by Claude Code.

Directory Structure

.claude/skills/web-fetch/
├── SKILL.md          # Main skill definition with instructions
├── REFERENCE.md      # Technical implementation details
├── examples.md       # Practical usage examples
└── README.md         # This file

Quick Start

Basic Usage

User: Can you fetch and analyze https://example.com/article

Claude will:
1. Validate the URL (user-provided ✓)
2. Execute WebFetch with specific prompt
3. Track fetch count (1/10)
4. Analyze and present results

With Domain Filtering

User: Fetch Python docs but only from docs.python.org

Claude will:
1. Apply allowed_domains filter
2. Fetch from approved domain only
3. Reject any external links automatically

Research Workflow

User: Find and analyze articles about Rust async

Claude will:
1. WebSearch for relevant articles
2. Present results to user
3. Ask which to fetch (no assumptions)
4. Fetch approved URLs with tracking
5. Synthesize findings

Security Controls

1. URL Validation

• ✅ User-provided URLs
• ✅ Search result URLs
• ✅ Local file URLs (with confirmation)
• ❌ Generated URLs (NEVER)
• ❌ Guessed URLs (NEVER)

2. Domain Filtering

Whitelist (allowed_domains):

Only fetch from approved domains:
["docs.python.org", "peps.python.org"]

Blacklist (blocked_domains):

Block specific domains:
["spam-site.com", "unreliable-source.net"]

3. Max Uses Tracking

Default Limits:

• Limit: 10 fetches/conversation
• Warning: 7 fetches (70% threshold)
• Hard stop: Requires approval at 10+

Prevents:

• Data exfiltration via repeated requests
• Unintended access to sensitive resources
• Excessive automated fetching

Usage Patterns

Pattern 1: Single URL Fetch

User provides URL → Claude fetches → Analyzes → Presents results
Fetches: 1

Pattern 2: Search + Selective Fetch

Search for topic → Present results → User selects → Fetch selected → Synthesize
Fetches: 1-3 (selective)

Pattern 3: Comparative Analysis

Fetch source A → Fetch source B → Compare → Synthesize findings
Fetches: 2-4

Pattern 4: Deep Research

Main article → Related refs → User-approved deep dive → Comprehensive analysis
Fetches: 3-7

When to Use This Skill

Use web-fetch when:

• ✅ User provides a specific URL to analyze
• ✅ Following up on WebSearch results
• ✅ Fetching documentation or tutorials
• ✅ Analyzing PDF research papers
• ✅ Comparing information across multiple sources
• ✅ Extracting structured data from web pages

Don't use when:

• ❌ URL would need to be generated/guessed
• ❌ Content is behind authentication
• ❌ User hasn't provided explicit URL
• ❌ Trying to access internal/private resources without authorization

WebFetch Tool Parameters

WebFetch(
  url: string,                    // Required: user-provided URL
  prompt: string,                 // Required: specific extraction instructions
  allowed_domains?: string[],     // Optional: whitelist
  blocked_domains?: string[]      // Optional: blacklist
)

Files Description

SKILL.md

The main skill file with:

• Metadata (name, description, version, allowed-tools)
• When to use this skill
• Security controls explanation
• Usage guidelines and workflows
• Error handling procedures
• Best practices and checklist

REFERENCE.md

Technical implementation details:

• WebFetch API specification
• Security architecture
• Domain filtering implementation
• Advanced usage patterns
• Edge cases and solutions
• Performance considerations
• Troubleshooting guide

examples.md

Real-world usage examples:

• Fetching documentation
• Research workflows
• PDF analysis
• Multi-source comparison
• Handling fetch limits
• Domain filtering scenarios
• Redirect management

Best Practices

✅ Do

1. Always use user-provided or search-returned URLs 2. Track fetch count after every operation 3. Warn user at 70% of limit (7/10) 4. Apply domain filters when security matters 5. Write specific, actionable prompts 6. Present findings in structured format 7. Offer relevant follow-up options

❌ Don't

1. Never generate or guess URLs 2. Never bypass domain filters without approval 3. Never auto-fetch without user intent 4. Never exceed limits without explicit permission 5. Never use vague prompts ("tell me about this") 6. Never fetch sensitive/internal URLs without authorization

Example Prompts

Good Prompts ✓

"Extract installation instructions and list all dependencies"
"Summarize main argument and supporting evidence"
"List API endpoints with parameters and examples"
"Extract code examples showing async/await usage"
"Compare approach A vs approach B from these two sources"

Poor Prompts ✗

"Tell me about this page" (too vague)
"What does this say?" (unclear goal)
"Read this" (no analysis specified)
"Everything" (overly broad)

Troubleshooting

Advanced Topics

Redirect Handling

When a URL redirects: 1. WebFetch detects redirect 2. Returns redirect destination URL 3. Claude informs user 4. Asks for confirmation 5. Fetches destination (counts as +1)

Fetch Limit Management

fetch_count = 0

After each WebFetch:
  fetch_count += 1

At 7: Warn user (approaching limit)
At 10: Require explicit approval
Beyond 10: Continue with documented approval

Cache Behavior

• Duration: 15 minutes
• Self-cleaning (automatic expiration)
• Same URL within 15min = cached (no fetch count increment)
• After 15min = new fetch required

Integration with Other Skills

Works well with:

• summarization: Fetch → summarize content
• code-review: Fetch → analyze code examples
• research: Multi-source information gathering
• documentation: Fetch official docs → extract specific info

Version History

• v1.0.0 (2024): Initial release
• Basic WebFetch functionality
• Domain filtering (whitelist/blacklist)
• Max uses tracking
• PDF support
• Redirect handling

Support

For issues or questions: 1. Check REFERENCE.md for technical details 2. Review examples.md for usage patterns 3. Consult Claude Code documentation 4. Report issues to skill maintainer

License

Part of Claude Code Skills. See project license.

---

Quick Reference:

Remember: Security through URL validation, domain filtering, and usage limits.

Web Fetch Skill - Technical Reference

Overview

This document provides technical implementation details, edge cases, and advanced usage patterns for the web-fetch skill.

WebFetch Tool API

Function Signature

WebFetch(
  url: string,              // Required: fully-formed valid URL
  prompt: string,           // Required: analysis instructions
  allowed_domains?: string[], // Optional: whitelist domains
  blocked_domains?: string[]  // Optional: blacklist domains
)

Parameters

url (required)

• Type: string (URI format)
• Constraints:
• Must be fully-formed (include protocol)
• HTTP automatically upgraded to HTTPS
• Must be user-provided or from search results
• Cannot be generated or guessed
• Valid: https://example.com/page
• Invalid: example.com/page (missing protocol)

prompt (required)

• Type: string
• Purpose: Describes what information to extract
• Best practices:
• Be specific and actionable
• Focus on desired output format
• Include context if needed
• Avoid vague requests

allowed_domains (optional)

• Type: array of strings
• Behavior: Only domains in this list can be fetched
• Use case: Restricting to official documentation
• Example: ["docs.python.org", "peps.python.org"]
• Mutually exclusive with: blocked_domains

blocked_domains (optional)

• Type: array of strings
• Behavior: Domains in this list cannot be fetched
• Use case: Avoiding unreliable sources
• Example: ["spam-site.com", "malware-domain.net"]
• Mutually exclusive with: allowed_domains

Return Behavior

• Success: Returns processed content (HTML→Markdown + analysis)
• Redirect: Returns special redirect message with new URL
• Failure: Returns error message
• Large content: May be summarized automatically
• Cached: 15-minute self-cleaning cache for same URL

Security Architecture

Data Exfiltration Prevention

The max_uses limit prevents malicious attempts to exfiltrate data through web requests:

Attack Vector

An attacker could try to: 1. Encode sensitive data in URL parameters 2. Make repeated requests to external server 3. Exfiltrate information via HTTP logs

Mitigation Strategy

Max Uses Limit:
├── Default: 10 fetches/conversation
├── Warning: 7 fetches (70% threshold)
├── Hard Stop: 10 fetches (requires approval)
└── Reset: Only on new conversation

Implementation Pseudocode

fetch_count = 0
MAX_USES = 10
WARN_THRESHOLD = 7

def web_fetch_with_tracking(url, prompt, **kwargs):
    global fetch_count

    # Check limit
    if fetch_count >= MAX_USES:
        user_approval = ask_user("Fetch limit reached. Continue?")
        if not user_approval:
            return "Fetch cancelled by user"

    # Warn at threshold
    if fetch_count == WARN_THRESHOLD:
        notify_user(f"Approaching limit ({fetch_count}/{MAX_USES})")

    # Execute fetch
    result = WebFetch(url, prompt, **kwargs)
    fetch_count += 1

    return result

URL Validation

Never generate URLs to prevent:

• Accessing unintended resources
• Data exfiltration through crafted URLs
• Privacy violations

Acceptable URL sources: 1. Direct user input: "Fetch https://example.com" 2. Search results: URLs returned by WebSearch 3. Local files: URLs found in code/docs (with confirmation)

Unacceptable URL sources: 1. Generated from patterns 2. Inferred from partial input 3. Assumed from context 4. Constructed from templates

Domain Filtering Implementation

Whitelist Approach (allowed_domains)

When to use:

• Restricting to official documentation
• Corporate security policies
• High-trust sources only

Example scenarios:

# Scenario 1: Python documentation only
WebFetch(
    url="https://docs.python.org/3/library/asyncio.html",
    prompt="Explain asyncio event loop",
    allowed_domains=["docs.python.org", "peps.python.org"]
)

# Scenario 2: Academic research only
WebFetch(
    url="https://arxiv.org/pdf/2301.12345.pdf",
    prompt="Extract methodology",
    allowed_domains=["arxiv.org", "scholar.google.com", "ieee.org"]
)

# Scenario 3: Corporate intranet
WebFetch(
    url="https://wiki.company.com/engineering/guides",
    prompt="List all deployment guides",
    allowed_domains=["wiki.company.com", "docs.company.com"]
)

Blacklist Approach (blocked_domains)

When to use:

• Avoiding known unreliable sources
• Blocking competitors
• Preventing access to problematic sites

Example scenarios:

# Scenario 1: Avoid content farms
WebFetch(
    url="https://quality-source.com/article",
    prompt="Analyze this tutorial",
    blocked_domains=["spam-site.com", "low-quality-blog.net"]
)

# Scenario 2: Research without opinion pieces
WebFetch(
    url="https://news-site.com/tech-analysis",
    prompt="Extract factual information",
    blocked_domains=["opinion-blog.com", "editorial-site.com"]
)

Filter Validation

Priority order: 1. If allowed_domains specified → only those domains allowed 2. If blocked_domains specified → all except those domains allowed 3. If both specified → ERROR (mutually exclusive) 4. If neither specified → all domains allowed (default)

Advanced Usage Patterns

Pattern 1: Comparative Analysis

Fetch multiple sources and synthesize:

Task: Compare error handling in Python vs Rust

Steps:
1. WebFetch(
     "https://docs.python.org/3/tutorial/errors.html",
     "Explain Python exception handling with examples",
     allowed_domains=["docs.python.org"]
   ) → fetch_count = 1

2. WebFetch(
     "https://doc.rust-lang.org/book/ch09-00-error-handling.html",
     "Explain Rust error handling with Result and Option types",
     allowed_domains=["doc.rust-lang.org"]
   ) → fetch_count = 2

3. Synthesize comparison table:
   | Aspect | Python | Rust |
   |--------|--------|------|
   | Model  | Exceptions | Result/Option |
   | ...    | ...    | ... |

Pattern 2: Research Pipeline

Search → Filter → Fetch → Analyze:

Task: Research best practices for API authentication

Steps:
1. WebSearch("API authentication best practices 2024")
   → Returns 10 results

2. Present results to user
   → User selects 3 relevant URLs

3. Sequential fetches:
   For each URL:
   - WebFetch(url, "Extract authentication methods and security considerations")
   - fetch_count += 1
   - Accumulate findings

4. Synthesize comprehensive guide from all sources

Pattern 3: Deep Dive Analysis

Fetch, analyze, then fetch related resources:

Task: Understand a complex technical concept

Steps:
1. WebFetch(main_article_url,
     "Extract main concepts, related topics, and references"
   ) → fetch_count = 1

2. Identify 2-3 key related topics from references

3. Ask user: "I found references to X, Y, Z. Should I fetch those too?"

4. If approved:
   For each related topic:
   - WebFetch(related_url, "Explain this related concept")
   - fetch_count += 1

5. Build comprehensive understanding with cross-references

Pattern 4: PDF Document Chain

Fetch and analyze academic papers with citations:

Task: Analyze ML paper and key citations

Steps:
1. WebFetch("https://arxiv.org/pdf/main-paper.pdf",
     "Extract abstract, methods, results, and top 3 cited papers"
   ) → fetch_count = 1

2. User reviews, selects citation #2 to explore

3. WebFetch("https://arxiv.org/pdf/cited-paper.pdf",
     "Extract methodology relevant to [main paper's approach]"
   ) → fetch_count = 2

4. Compare and contrast methodologies

Edge Cases and Solutions

Edge Case 1: Redirect Chain

Problem: URL redirects multiple times Solution:

1. First WebFetch returns redirect to URL_2
2. Inform user: "Redirected to URL_2. Fetching..."
3. Second WebFetch(URL_2) → fetch_count = 2
4. If URL_2 also redirects, inform user and ask to continue
5. Track each redirect as separate fetch

Edge Case 2: Fetch Limit During Multi-Fetch Operation

Problem: Reach limit mid-way through comparative analysis Solution:

Scenario: Comparing 5 frameworks, limit is 10, currently at 8

1. Fetch framework 1 → count = 9
2. Fetch framework 2 → count = 10 (limit reached)
3. Stop and inform user: "Reached limit after 2/5 frameworks"
4. Options:
   a) Present partial analysis
   b) Ask to continue (with approval)
   c) Save remaining URLs for next conversation

Edge Case 3: Invalid URL After Redirect

Problem: Redirect leads to broken/invalid URL Solution:

1. WebFetch returns redirect to invalid URL
2. Detect failure (not just redirect)
3. Inform user: "Redirect failed - destination unavailable"
4. Don't count failed fetch against limit (only successful fetches)
5. Suggest alternative approach

Edge Case 4: Domain Filter Conflict

Problem: User-provided URL blocked by filter Solution:

User: "Fetch https://medium.com/article"
(but blocked_domains includes medium.com)

Response:
1. Detect conflict before fetching
2. Inform user: "URL blocked by domain filter"
3. Ask: "Override filter for this fetch?"
4. If yes: fetch without filter, count = +1
5. If no: suggest alternative source

Edge Case 5: Extremely Large Content

Problem: Fetching very large page/PDF Solution:

1. WebFetch tool auto-summarizes large content
2. Receive summarized version
3. Inform user: "Content was large - received summary"
4. If user needs more detail:
   - Suggest specific section to re-fetch
   - Or use more targeted prompt
5. Each attempt counts as separate fetch

Edge Case 6: Dynamic/JavaScript Content

Problem: Content requires JavaScript execution Solution:

1. WebFetch returns HTML without JS-rendered content
2. Detect missing expected content
3. Inform user: "Page may require JavaScript"
4. Alternatives:
   a) Suggest searching for static version
   b) Suggest API endpoint if available
   c) Suggest cached/archive version

Performance Considerations

Caching Strategy

• Duration: 15 minutes
• Behavior: Self-cleaning (automatic expiration)
• Benefit: Faster response for repeated URLs
• Trade-off: May show stale content for rapidly updating pages

When cache helps:

1. User: "Fetch example.com/docs"
   → Fetches and caches, count = 1

2. User: "Can you explain section 3 from that page?"
   → Uses cache, count = 1 (not incremented)

3. 20 minutes later...
   → Cache expired, new fetch needed

Parallel vs Sequential Fetches

Sequential (recommended):

For each URL in [url1, url2, url3]:
    result = WebFetch(url, prompt)
    fetch_count += 1
    analyze(result)
    present_to_user()

Parallel (not recommended):

# Don't do this - harder to track and analyze
results = parallel_map(urls, lambda url: WebFetch(url, prompt))
fetch_count += len(urls)

Reasoning: Sequential allows incremental presentation and better user control.

Error Handling Reference

Testing Scenarios

Test 1: Basic Fetch

Input: User provides single URL
Expected: Fetch succeeds, count = 1
Verify: Content returned and analyzed

Test 2: Domain Whitelist

Input: URL + allowed_domains list
Expected: Only whitelisted domains work
Verify: Other domains rejected pre-fetch

Test 3: Limit Enforcement

Input: 11 fetch requests
Expected: Warning at 7, stop at 10
Verify: Requires approval for 11th

Test 4: Redirect Handling

Input: URL that redirects
Expected: Detect redirect, fetch new URL
Verify: Both URLs counted separately

Test 5: PDF Processing

Input: PDF URL
Expected: Extracts text and visual content
Verify: Returns page-by-page analysis

Test 6: Cache Behavior

Input: Same URL twice within 15 min
Expected: Second fetch uses cache
Verify: Count only incremented once

Integration Examples

With Summarization Skill

1. WebFetch(article_url, "Extract full content")
2. Pass to summarization skill
3. Return concise summary

With Code Review Skill

1. WebFetch(github_url, "Extract code examples")
2. Pass to code review skill
3. Analyze code quality and patterns

With Learning Path Creation

1. WebSearch("learn topic X")
2. WebFetch top 3 tutorial URLs
3. Synthesize learning path with:
   - Beginner → Intermediate → Advanced
   - Resources from fetched content

Troubleshooting Guide

Best Practices Checklist

Before each WebFetch:

• [ ] URL is from valid source (user/search/file)
• [ ] Prompt is specific and actionable
• [ ] fetch_count checked against limit
• [ ] Domain filters appropriate for task
• [ ] User aware of what will be fetched
• [ ] Ready to handle redirect if occurs
• [ ] Plan for analyzing results
• [ ] Consider if cache may apply

After each WebFetch:

• [ ] Increment fetch_count
• [ ] Check if approaching limit (warn at 7)
• [ ] Analyze results thoroughly
• [ ] Present findings clearly
• [ ] Determine if additional fetches needed
• [ ] Update user on progress

Configuration Recommendations

Conservative Settings (High Security)

max_uses: 5
warn_threshold: 3
allowed_domains: ["docs.python.org", "trusted-site.com"]
require_approval: true (for all fetches)

Balanced Settings (Recommended)

max_uses: 10
warn_threshold: 7
domain_filtering: case-by-case
require_approval: false (until limit)

Research-Intensive Settings

max_uses: 20
warn_threshold: 15
blocked_domains: ["spam-site.com", ...]
require_approval: false (until limit)

API Limits and Rate Limiting

WebFetch Tool Limits:

• No explicit rate limit per se
• max_uses enforced at skill level
• Respect site-specific robots.txt
• 15-minute cache reduces redundant requests

Site-Specific Considerations:

• Some sites block automated requests
• Academic sites (arXiv) usually allow
• News sites may have paywalls
• APIs may have separate rate limits

Future Enhancements

Potential improvements: 1. Dynamic limit adjustment based on task type 2. Persistent fetch history across conversations 3. Smart caching with content change detection 4. Batch fetch optimization for multiple URLs 5. Domain reputation scoring for auto-filtering 6. Integration hooks for custom post-processing

---

Quick Reference Card

╔════════════════════════════════════════════════════════╗
║             WEB FETCH SKILL QUICK REFERENCE             ║
╠════════════════════════════════════════════════════════╣
║ Basic Usage:                                            ║
║   WebFetch(url, prompt)                                 ║
║                                                         ║
║ With Whitelist:                                         ║
║   WebFetch(url, prompt, allowed_domains=[...])          ║
║                                                         ║
║ With Blacklist:                                         ║
║   WebFetch(url, prompt, blocked_domains=[...])          ║
║                                                         ║
║ Limits:                                                 ║
║   • Default: 10 fetches/conversation                    ║
║   • Warning: 7 fetches (70%)                            ║
║   • Requires approval: 11+ fetches                      ║
║                                                         ║
║ URL Sources (Valid):                                    ║
║   ✓ User-provided                                       ║
║   ✓ Search results                                      ║
║   ✓ Local files (confirmed)                             ║
║   ✗ Generated/guessed                                   ║
║                                                         ║
║ Supported Content:                                      ║
║   • HTML (converted to Markdown)                        ║
║   • PDF (text + visual extraction)                      ║
║   • Cached (15-minute TTL)                              ║
║                                                         ║
║ Error Handling:                                         ║
║   • Redirects: Follow with new fetch (+1 count)         ║
║   • Failures: Don't retry auto (still counts)           ║
║   • Large content: Auto-summarized                      ║
╚════════════════════════════════════════════════════════╝