Nextjs Seo

# AI Search Optimization (GEO / AEO) & AI Crawlers

How to make a Next.js site visible in AI answer engines (Google AI Overviews / AI Mode, ChatGPT Search, Perplexity, Gemini, Claude) — grounded in what's actually established, not hype.

## Contents

- Core principle (layer on top of SEO, not a replacement)
- Google AI Overviews / AI Mode — official eligibility
- llms.txt — honest status
- AI crawlers: training vs search/citation bots
- Recommended robots.ts pattern
- robots.txt is advisory (WAF/edge for real blocking)
- Content structure that drives AI citation
- Structured data for AI search
- Measuring AI search visibility

## Core principle

**GEO** (Generative Engine Optimization) / **AEO** (Answer Engine Optimization) = optimizing so your content is **cited** in AI-generated answers. It is a layer **on top of** classic SEO, not a replacement. Most tactics that help AI citation also help users and traditional search — favor those low-risk, high-overlap moves over AI-specific "tricks."

## Google AI Overviews / AI Mode — official eligibility

Per Google's official guidance: there are **no special requirements** to appear in AI Overviews or AI Mode.

> "You don't need to create new machine readable files, AI text files, or markup to appear in these features. There's also no special schema.org structured data that you need to add."

Eligibility = pages that are **indexed and eligible to be shown in Google Search with a snippet**. So the baseline is normal technical SEO: crawlable, indexable, server-rendered content. (Source: developers.google.com/search/docs/appearance/ai-features)

**Do not promise ranking/citation gains from any "AI-specific file or schema."** Google says none are needed.

## llms.txt — honest status

`llms.txt` is a **community proposal**, not an adopted standard:

- **Not supported by Google.** John Mueller publicly compared it to the discredited `keywords` meta tag.
- **Negligible real usage by AI crawlers**, and no demonstrated correlation between having an `llms.txt` and being cited in AI answers.
- **The one legitimate, working use case:** documentation / developer-tool sites whose users paste docs into IDE/coding agents (Cursor, Claude Code, Copilot, Cline). Those agents do look for `/llms.txt` and `/llms-full.txt`. Next.js itself ships `https://nextjs.org/docs/llms.txt`.

So: recommend `llms.txt` **only** for docs/dev-tool sites as an AI-assistant ergonomics nicety — **not** as an SEO/citation ranking tactic. There is no `MetadataRoute` helper; implement as a Route Handler:

```typescript
// app/llms.txt/route.ts
export const dynamic = 'force-static';

export function GET() {
  const body = `# Site Name\n\n> One-line summary.\n\n## Docs\n- [Getting started](https://your-site.com/docs)\n`;
  return new Response(body, { headers: { 'Content-Type': 'text/plain' } });
}
```

## AI crawlers: training vs search/citation bots

The key 2026 concept: **separate training crawlers from search/citation crawlers.**

- **Training crawlers** collect content to train models. Blocking them opts you out of training and costs you no referral traffic.
- **Search / citation crawlers** fetch content in real time to power AI answers. **Blocking them removes you from that engine's AI answers** — now a high-value channel.
- **User-initiated fetchers** retrieve a page a user explicitly asked the assistant about. Blocking them means the AI can't read a page the user pointed it to.
- **Training opt-out tokens** (`Google-Extended`, `Applebot-Extended`) are robots.txt tokens, **not crawlers** — they opt you out of Gemini/Apple-Intelligence *training* with **no effect on Search ranking or AI Overviews** (AI Overviews use Googlebot, so you can't leave AI Overviews via robots.txt without leaving Search).

| Vendor | Training | Search / citation | User-initiated | Training opt-out token |
|--------|----------|-------------------|----------------|------------------------|
| OpenAI | `GPTBot` | `OAI-SearchBot` | `ChatGPT-User` | — |
|