Llms Txt Checker
Audit a site’s robots.txt, llms.txt, and llms-full.txt so AI coding tools and answer engines can discover your docs correctly.
Overview
LLMs.txt Checker is an agent skill most often used in Launch (also Build docs, Grow content) that audits robots.txt, llms.txt, and llms-full.txt on a domain and reports AI-readiness findings with fixes.
Install
npx skills add https://github.com/infrasity-labs/dev-gtm-claude-skills --skill llms-txt-checkerWhat is this skill?
- Normalizes a domain or URL and probes robots.txt, llms.txt, and llms-full.txt via curl
- Scores each file against a structured checklist covering structure, completeness, and AI-readiness signals
- Returns a formatted report with pass, warn, and fail findings plus actionable fixes
- Targets DevRel and technical writers doing pre-launch or post-redesign AI discoverability validation
- Activates from /dev-gtm llms-txt-check or natural-language requests about llms.txt and AI-readiness
- Probes 3 AI-readiness files: robots.txt, llms.txt, and llms-full.txt
- Structured checklist with pass/warn/fail report formatting
Adoption & trust: 1 installs on skills.sh; 24 GitHub stars; trending (+100% hot-view momentum).
What problem does it solve?
You are shipping or promoting technical docs but do not know whether AI agents can find structured llms.txt guidance or whether robots and cross-file references are correct.
Who is it for?
Solo builders and small DevRel teams validating docs.example.com-style properties before launch, GEO experiments, or agent-tooling announcements.
Skip if: Teams that only need on-page keyword SEO without llms.txt/robots governance, or sites where you cannot allow outbound HTTP fetches to the target domain.
When should I use this skill?
You provide a domain or URL and ask about llms.txt, llms-full.txt, robots.txt, or AI-readiness for docs.
What do I get? / Deliverables
You receive a scored audit across three probed files with pass/warn/fail detail and specific fixes so you can publish compliant llms.txt assets before distribution and GEO work.
- Formatted AI-readiness audit report
- Per-file pass/warn/fail findings
- Actionable remediation list
Recommended Skills
Journey fit
Spans multiple journey phases - primary shelf plus alternate fits below.
Canonical shelf is Launch because the skill validates GEO/AEO file presence and structure—the same artifacts teams fix before AI-search and agent discoverability campaigns. GEO is the facet for llms.txt, AEO signals, and AI crawler readiness checks rather than classic keyword SEO alone.
Where it fits
Run an audit right after adding llms.txt to a docs site repo to confirm structure before merging.
Validate llms-full.txt completeness before announcing agent-friendly documentation.
Re-check the domain after a major docs IA change to ensure AI crawlers still see consistent file references.
Probe a marketing subdomain’s llms.txt before scaling paid distribution to AI-native audiences.
How it compares
Use this checker for live llms.txt/robots probes and structured scoring—not as a replacement for writing llms-full content or a full technical SEO crawl suite.
Common Questions / FAQ
Who is llms-txt-checker for?
Developer marketers, GEO/AEO practitioners, and technical writers who need a fast, repeatable audit that AI coding tools can actually discover their documentation.
When should I use llms-txt-checker?
At Launch/GEO before campaigns; during Build/docs when adding llms.txt to a repo; at Grow/content when refreshing docs discoverability; anytime you have a domain and ask about llms.txt or AI-readiness.
Is llms-txt-checker safe to install?
Review the Security Audits panel on this Prism page and treat the skill as network-active—it fetches public URLs on domains you specify via curl-like requests.
SKILL.md
READMESKILL.md - Llms Txt Checker
# LLMs.txt Checker Audits any domain's AI-readiness by probing `robots.txt`, `llms.txt`, and `llms-full.txt` directly, then scores each file against a structured checklist and delivers a formatted report with pass/warn/fail findings and actionable fixes. --- ## What this skill does You give it a domain or URL. The skill normalises the input, fetches all three AI-readiness files via `curl`, and audits each one against a structured checklist covering structure, content completeness, and AI-readiness signals. It then reports what was found, what was missing, and exactly what to fix. Built for: - **Developer marketing and DevRel teams** checking whether their docs are discoverable by AI tools like Claude Code, Cursor, and GitHub Copilot - **GEO / AEO practitioners** validating that `llms.txt` and `llms-full.txt` are correctly structured and referenced - **Technical writers** doing a pre-launch AI-readiness check before a docs redesign --- ## Installation ### Claude Code (Recommended) Clone the repo. The skill activates automatically when you open it in Claude Code: ```bash git clone https://github.com/Infrasity-Labs/dev-gtm-claude-skills.git cd dev-gtm-claude-skills claude ``` Then trigger it with: ``` /dev-gtm llms-txt-check docs.example.com ``` Or just describe what you want. Claude activates the skill when you provide a domain and ask about `llms.txt` or AI-readiness. ### Claude Web (Free / Pro) 1. Go to **[Settings → Capabilities](https://claude.ai/settings/capabilities)** and enable **Code execution and file creation** 2. Go to **[Customize → Skills](https://claude.ai/customize/skills)** 3. Click **+** → **Create skill** → **Upload a skill** 4. Zip this skill folder and upload it: cd dev-gtm-claude-skills/skills/llms-txt-checker zip -r ../llms-txt-checker.zip . zip -r llms-txt-checker.zip llms-txt-checker/ ``` Upload `llms-txt-checker.zip` and toggle it on. > **Egress network required.** This skill uses curl to make outbound HTTP requests. In Claude Code, you will be prompted to allow network access when the skill runs. In Claude Web, code execution (enabled above) covers this automatically. --- ## How to use ``` Check llms.txt for docs.anthropic.com ``` ``` Does stripe.com have llms.txt? ``` ``` Audit the AI-readiness of docs.example.com ``` ``` /dev-gtm llms-txt-check linear.app ``` Claude accepts any phrasing that includes a domain and mentions `llms.txt`, AI-readiness, GEO, or AEO. --- ## Inputs | Field | Required | Notes | |-------|----------|-------| | Domain or URL | ✅ | Any format: `example.com`, `https://docs.example.com/en/home`, etc. Claude normalises it | --- ## Output A structured audit report rendered directly in chat: ``` ## LLMs.txt Audit: docs.example.com ### Discovery llms.txt: ✅ 200 (4.2 KB) llms-full.txt: ❌ 404 robots.txt: ✅ 200 ### llms.txt: ✅ Found ✅ H1 title present ✅ Blockquote summary below H1 ✅ H2 sections grouping links ❌ Links use relative URLs (must be absolute) ❌ No reference to llms-full.txt ### llms-full.txt: ❌ Not Found ### robots.txt Signal ✅ User-agent: * with Allow: / ✅ ai-train=no (acceptable) ### Summary & Recommendations 1. Switch all link URLs to absolute paths 2. Create llms-full.txt and reference it from llms.txt 3. ... ``` --- ## Things to know **Redirects are followed automatically.** The skill uses `curl -L`, so 301/302 redirects are transparent; the final destination URL is what gets audited. **404 ≠ broken site.** A 404 on `llms.txt` simply means the file is not published at the standard path. The skill always checks whether the file is referenced from a non-standard location before marking it absent. **Mintlify and Fern auto-generate both files.** If the site is on Mintlify or Fern, both `llms.txt` and `llms-full.txt` are auto-generated. The skill notes this in the discovery section. **No API keys needed.** This skill runs entirely on `curl`: no Ahrefs, no DataForSEO, no third-party services. --- ## How it works 1. **Normalise th