Sitemap

SEO — Sitemap

Overview

A Sitemap is an XML file (or RSS/Atom/text alternative) that lists URLs on your site so search engines can discover and crawl them more efficiently. It supplements link-based discovery; it does not guarantee indexing.

Principles: UTF-8 encoding; absolute URLs with protocol; one host per sitemap; accurate lastmod; stay under size limits; place at site root when possible.

Reference: sitemaps.org protocol · FAQ

---

When to use a sitemap

Pair with robots.txt (Sitemap: https://example.com/sitemap.xml) and Search Console submission.

---

XML format (protocol 0.9)

Required structure:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://www.example.com/</loc>
    <lastmod>2025-06-13</lastmod>
    <changefreq>monthly</changefreq>
    <priority>0.8</priority>
  </url>
</urlset>

Entity escaping in XML: & → &, < → <, > → >, " → ", ' → '.

---

Limits (FAQ)

• Max 50,000 URLs per sitemap file.
• Max 50 MB uncompressed (52,428,800 bytes).
• Use gzip compression when serving (file must still be ≤50 MB uncompressed).
• Exceed limits → split into multiple sitemaps + sitemap index.

Sitemap index

<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <sitemap>
    <loc>https://www.example.com/sitemap-posts.xml</loc>
    <lastmod>2025-06-13T12:00:00+00:00</lastmod>
  </sitemap>
  <sitemap>
    <loc>https://www.example.com/sitemap-pages.xml</loc>
    <lastmod>2025-06-01</lastmod>
  </sitemap>
</sitemapindex>

Index: max 50,000 sitemaps, max 50 MB. Index lastmod = when that sitemap file changed (enables incremental crawl).

---

File location rules

• Recommended: https://example.com/sitemap.xml (root).
• All <loc> URLs must share the same protocol and host as the sitemap (e.g. no http URLs in an https sitemap).
• Subpath sitemap (/catalog/sitemap.xml) may only list URLs under /catalog/.
• List one canonical version of each URL (not both http and https).
• No session IDs in URLs.
• Include frameset and frame content URLs if site uses frames (legacy).

---

Next.js (App Router)

Use app/sitemap.ts exporting default async function returning MetadataRoute.Sitemap:

import type { MetadataRoute } from 'next'

const SITE = 'https://example.com'

export default async function sitemap(): Promise<MetadataRoute.Sitemap> {
  const staticPages: MetadataRoute.Sitemap = [
    { url: `${SITE}/`, lastModified: new Date(), priority: 1 },
    { url: `${SITE}/about`, lastModified: new Date(), priority: 0.8 },
  ]

  const posts = await fetchPostSlugs()
  const postPages = posts.map((slug) => ({
    url: `${SITE}/blog/${slug}`,
    lastModified: new Date(),
    priority: 0.7,
  }))

  return [...staticPages, ...postPages]
}

• Next.js serves at /sitemap.xml automatically.
• Use real lastModified from CMS when available.
• For multiple sitemaps, use generateSitemaps (Next.js 13.3+) or route handlers.

robots.txt

User-agent: *
Allow: /

Sitemap: https://example.com/sitemap.xml

---

After publishing

1. Validate XML against sitemap.xsd. 2. Submit in Google Search Console / Bing Webmaster Tools. 3. Add Sitemap: line to robots.txt. 4. Monitor crawl stats; update lastmod when content actually changes.

---

vs JSON-LD and llms.txt

Use all three (sitemap, json-ld, llms-txt) for a complete discoverability stack.

---

Common mistakes

• Wrong lastmod — sitemap generation date instead of page update date.
• Relative URLs — must include https://.
• Mixed hosts — subdomain URLs in www sitemap.
• Session IDs in <loc>.
• Duplicate http/https entries for same page.
• Priority 1.0 everywhere — dilutes relative signaling.
• Unescaped `&` in query strings — breaks XML.
• Oversized file — split before hitting 50k / 50MB.

---

Related skills

• json-ld — structured data on pages listed in the sitemap.
• llms-txt — llms.txt for AI assistants (curated, not a sitemap replacement).

---

Additional resources

• reference.md — Protocol, FAQ, schemas, Search Console, Next.js.
• Protocol: https://www.sitemaps.org/protocol.html
• FAQ: https://www.sitemaps.org/faq.php

License

Skill License - Free Use

This skill (the documentation, structure, and implementation) was created by Ender Puentes <Endev/> and is provided for free and open use. You are free to:

• Use this skill in any project, personal or commercial
• Modify the skill to fit your needs
• Distribute the skill to others
• Share modified versions of the skill
• Include this skill in your own skill collections

No restrictions apply – this skill is available for unrestricted use. Attribution is appreciated but not required.

Note: This skill (sitemap) documents the Sitemap protocol (sitemaps.org); the skill itself is an independent work. The sitemaps.org specification has its own terms (Attribution-ShareAlike Creative Commons).

Sitemap — Reference & Official Documentation

This file complements the sitemap skill with official documentation links and extra context for indexing.

Official specification

• Home: https://www.sitemaps.org/
• Protocol (XML format): https://www.sitemaps.org/protocol.html
• FAQ: https://www.sitemaps.org/faq.php
• Terms: https://www.sitemaps.org/terms.php
• Version: Sitemap 0.90 (Attribution-ShareAlike CC; supported by Google, Bing, Yahoo historically)

XML schemas

• urlset: http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd
• sitemapindex: http://www.sitemaps.org/schemas/sitemap/0.9/siteindex.xsd

Required tags (urlset)

Namespace: xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"

Optional per URL: lastmod, changefreq, priority.

changefreq values

always, hourly, daily, weekly, monthly, yearly, never — hints only, not crawl commands.

Size limits (FAQ)

Compress with gzip for transfer; uncompressed size still applies.

lastmod (FAQ)

• W3C Datetime: 2004-09-22T14:12:14+00:00 or date-only 2004-09-22.
• Static files: actual file mtime.
• Dynamic pages: when underlying data changed (or best approximation).
• Include time portion if site changes frequently.

Entity escaping

URLs must also conform to RFC-3986 (URI) / RFC-3987 (IRI). Encoding: UTF-8.

Alternative formats (protocol)

• RSS 2.0 / Atom 0.3 / 1.0 — syndication feeds as sitemaps (recent URLs only).
• Plain text — one URL per line, UTF-8, fully qualified, max 50k lines / 50 MB.

Placement & scope (FAQ)

• Strongly recommended: https://example.com/sitemap.xml (root).
• All URLs in a sitemap must be on the same host as the sitemap file.
• Subpath sitemaps only include URLs under that path prefix.
• Cross-subdomain: use Search Console "cross submit" or separate properties per host.

Submission (FAQ)

After creating sitemap:

1. Submit to search engines (Search Console, Bing, ping). 2. Add to robots.txt: Sitemap: https://example.com/sitemap.xml

Does not guarantee inclusion in search results.

FAQ highlights

• priority does not change ranking vs other websites — only relative within site.
• URL order in file does not affect crawling.
• http vs https: list only one canonical version.
• Session IDs: remove from URLs.
• Frames: include both frameset and frame content URLs.

Google Search Central

• Sitemaps overview: https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap
• Search Console sitemap report: submit and monitor errors
• Google supports sitemap index, gzip, lastmod (treated as hint)

Next.js

• MetadataRoute.Sitemap: app/sitemap.ts → /sitemap.xml
• Fields: url, lastModified, changeFrequency, priority
• generateSitemaps + dynamic sitemap.ts for multiple sitemap files (large sites)
• robots.ts: app/robots.ts can reference sitemap URL

Comparison stack

Related skills

• json-ld — structured data on indexed pages.
• llms-txt — llms.txt for answer engines.