Baoyu Url To Markdown
Fetch almost any URL and turn it into clean markdown on disk using the baoyu-fetch CLI with site-specific adapters and Chrome CDP.
Overview
baoyu-url-to-markdown is an agent skill most often used in Idea (also Build, Grow) that fetches URLs and converts them to markdown via the baoyu-fetch CLI with Chrome CDP and site-specific adapters.
Install
npx skills add https://github.com/jimliu/baoyu-skills --skill baoyu-url-to-markdownWhat is this skill?
- baoyu-fetch CLI via Chrome CDP with vendored scripts and bun as the required runtime
- Built-in adapters for X/Twitter, YouTube transcripts, Hacker News threads, and generic pages through Defuddle
- Login and CAPTCHA flows via interaction wait modes when sites block headless fetch
- User-input tool priority (AskUserQuestion-style) with plain-text fallback when the runtime has no ask tool
- Outputs markdown suitable for notes, specs, and content pipelines from arbitrary URLs
- Built-in adapters for X/Twitter, YouTube transcripts, Hacker News, and generic pages via Defuddle
- Skill version 1.61.0 in SKILL metadata
Adoption & trust: 22.7k installs on skills.sh; 20.9k GitHub stars; 2/3 security scanners passed (skills.sh audits).
What problem does it solve?
You need a saved, agent-readable copy of a live webpage, thread, or transcript—not a bookmark—and generic fetch often breaks on JS, login, or site-specific layouts.
Who is it for?
Solo builders archiving research sources, social threads, video transcripts, or long articles into markdown using bun and a local Chrome-backed fetch CLI.
Skip if: Bulk crawling entire domains, non-URL ingestion, or teams that cannot install bun or run Chrome CDP locally.
When should I use this skill?
User wants to save a webpage as markdown, or needs URL fetch with login/CAPTCHA handling via baoyu-fetch.
What do I get? / Deliverables
You get clean markdown from the target URL (with optional user interaction for auth) ready to drop into notes, specs, or content drafts.
- Markdown file(s) representing the fetched URL content
- Captured transcripts or thread text where adapters apply
Recommended Skills
Journey fit
Spans multiple journey phases - primary shelf plus alternate fits below.
Canonical shelf is Idea → research because the primary trigger is saving web pages for reading, citation, and discovery before or alongside product work. Research subphase fits competitor pages, threads, transcripts, and articles you want as portable markdown rather than live tabs.
Where it fits
Pull a competitor’s pricing page into markdown before you scope your own offer.
Archive an X thread or HN discussion about a rival product for your validation notes.
Snapshot upstream API or framework docs into markdown beside your implementation plan.
Capture a published article or transcript as markdown to draft a newsletter or changelog post.
How it compares
A packaged URL-to-markdown workflow with site adapters—not a generic WebFetch one-liner or an MCP browser server.
Common Questions / FAQ
Who is baoyu-url-to-markdown for?
Indie builders and agent users who routinely turn web pages, X threads, YouTube transcripts, or HN discussions into markdown files for research and writing.
When should I use baoyu-url-to-markdown?
Use it in Idea/research when capturing competitors or sources, in Build/docs when pulling reference material into the repo, and in Grow/content when repurposing published pages—whenever the user asks to save a webpage as markdown.
Is baoyu-url-to-markdown safe to install?
It runs shell commands, network requests, and browser automation; review the Security Audits panel on this page and treat logged-in sessions and CAPTCHA flows as sensitive before installing.
SKILL.md
READMESKILL.md - Baoyu Url To Markdown
# URL to Markdown Fetches any URL via `baoyu-fetch` CLI (Chrome CDP + site-specific adapters) and converts it to clean markdown. ## User Input Tools When this skill prompts the user, follow this tool-selection rule (priority order): 1. **Prefer built-in user-input tools** exposed by the current agent runtime — e.g., `AskUserQuestion`, `request_user_input`, `clarify`, `ask_user`, or any equivalent. 2. **Fallback**: if no such tool exists, emit a numbered plain-text message and ask the user to reply with the chosen number/answer for each question. 3. **Batching**: if the tool supports multiple questions per call, combine all applicable questions into a single call; if only single-question, ask them one at a time in priority order. Concrete `AskUserQuestion` references below are examples — substitute the local equivalent in other runtimes. ## CLI Setup **Important**: The CLI source is vendored in `{baseDir}/scripts/lib`. `scripts/package.json` installs only third-party runtime dependencies. **Agent Execution Instructions**: 1. Determine this SKILL.md file's directory path as `{baseDir}` 2. Resolve `${BUN}` runtime: if `bun` installed → `bun`; else suggest installing Bun 3. If `{baseDir}/scripts/node_modules` does not exist, run `${BUN} install --cwd {baseDir}/scripts` 4. `${READER}` = `{baseDir}/scripts/baoyu-fetch` 5. Replace all `${READER}` in this document with the resolved value ## Preferences (EXTEND.md) Check EXTEND.md in priority order — the first one found wins: | Priority | Path | Scope | |----------|------|-------| | 1 | `.baoyu-skills/baoyu-url-to-markdown/EXTEND.md` | Project | | 2 | `${XDG_CONFIG_HOME:-$HOME/.config}/baoyu-skills/baoyu-url-to-markdown/EXTEND.md` | XDG | | 3 | `$HOME/.baoyu-skills/baoyu-url-to-markdown/EXTEND.md` | User home | | Result | Action | |--------|--------| | Found | Read, parse, apply settings | | Not found | **MUST** run first-time setup (see below) — do NOT silently create defaults | **EXTEND.md supports**: download media by default, default output directory. ### First-Time Setup ⛔ BLOCKING When EXTEND.md is not found, you **MUST** use `AskUserQuestion` to gather preferences before creating EXTEND.md. **NEVER** create EXTEND.md with silent defaults. Generation is BLOCKED until setup completes. Batch all three questions into a single call: - **Q1 — Media** (header "Media"): "How to handle images and videos in pages?" - "Ask each time (Recommended)" — Prompt after each save - "Always download" — Download to local `imgs/` and `videos/` - "Never download" — Keep remote URLs - **Q2 — Output** (header "Output"): "Default output directory?" - "url-to-markdown (Recommended)" — Save to `./url-to-markdown/{domain}/{slug}.md` - User may pick "Other" and type a custom path - **Q3 — Save** (header "Save"): "Where to save preferences?" - "User (Recommended)" — `~/.baoyu-skills/` (all projects) - "Project" — `.baoyu-skills/` (this project only) After answers, write EXTEND.md, confirm "Preferences saved to [path]", then continue. Full template: [references/config/first-time-setup.md](references/config/first-time-setup.md). ### Supported Keys | Key | Default | Values | Description | |-----|---------|--------|-------------| | `download_media` | `ask` | `ask` / `1` / `0` | `ask` = prompt each time, `1` = always, `0` = never | | `default_output_dir` | empty | path or empty | Default output directory (empty = `./url-to-markdown/`) | **EXTEND.md → CLI mapping**: | EXTEN