
Google Ai Mode Skill
- 44 installs
- 282 repo stars
- Updated January 8, 2026
- pleaseprompto/google-ai-mode-skill
google-ai-mode-skill is a Claude Code skill for ai & agent building.
About
google-ai-mode-skill is a Claude Code skill for ai & agent building. It helps solo builders move faster with AI-assisted coding.
- google-ai-mode-skill
- AI & Agent Building
- AI-coding skill
Google Ai Mode Skill by the numbers
- 44 all-time installs (skills.sh)
- +2 installs in the week ending Aug 4, 2026 (Skillselion tracking)
- Ranked #7,851 of 16,546 AI & Agent Building skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/pleaseprompto/google-ai-mode-skill --skill google-ai-mode-skillAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 44 |
|---|---|
| repo stars | ★ 282 |
| Last updated | January 8, 2026 |
| Repository | pleaseprompto/google-ai-mode-skill ↗ |
How do I helps with ai & agent building tasks during AI-assisted development.?
Helps with ai & agent building tasks during AI-assisted development.
Who is it for?
Best when you're working on ai & agent building and need structured help with google ai mode skill.
Skip if: Teams with no ai & agent building needs, or anyone wanting a generic chat assistant without this specific workflow.
When should I use this skill?
When you need to helps with ai & agent building tasks during AI-assisted development., or when google-ai-mode-skill is a claude code skill for ai & agent building.
What you get
Structured output aligned to google-ai-mode-skill: google-ai-mode-skill, AI & Agent Building.
Files
Google AI Mode Skill
Query Google's AI Search mode to retrieve comprehensive, source-grounded answers from across the web.
When to Use This Skill
Trigger this skill when the user:
- Requests current information beyond the knowledge cutoff (post-January 2025)
- Needs documentation or API references for libraries and frameworks
- Asks for coding examples or implementation patterns
- Wants technical comparisons or best practices
- Requires research with citations and sources
- Mentions "Google AI search", "Google AI mode", or "web research"
CLI Flags
Essential Flags
`--debug` - Enable comprehensive logging
python scripts/run.py search.py --query "..." --debug- Saves detailed logs to
logs/search_YYYY-MM-DD_HH-MM-SS.log - Logs every step: browser launch, CAPTCHA detection, AI content waiting, citation extraction
- Essential for troubleshooting CAPTCHA issues or failed searches
- Log file path printed at completion
`--save` - Save results to skill folder
python scripts/run.py search.py --query "..." --save- Saves markdown to
results/YYYY-MM-DD_HH-MM-SS_Query_Name.md - Timestamped filename for organized storage
- Results preserved in skill directory for future reference
- Use instead of
--outputfor automatic naming
Combined usage (recommended for debugging):
python scripts/run.py search.py --query "..." --debug --saveOther Flags
`--show-browser` - Show browser window (for CAPTCHA solving)
python scripts/run.py search.py --query "..." --show-browser`--output <path>` - Custom output file path
python scripts/run.py search.py --query "..." --output result.md`--json` - Include JSON metadata in output
python scripts/run.py search.py --query "..." --output result.md --jsonQuery Optimization Strategy
CRITICAL: Always optimize user queries before execution. Google AI Mode's quality depends on query precision.
Optimization Template
[Technology/Topic] [Version] [Year] ([Specific Aspect 1], [Aspect 2], [Aspect 3]). [Output format request].Optimization Rules
1. Include Current Year (2026) for up-to-date results 2. Use parentheses to list specific aspects needed 3. Request structured output (tables, comparisons, categorized lists) 4. Include version numbers for library/framework queries
Examples
| User Query | Optimized Query |
|---|---|
| "React hooks" | "React hooks best practices 2026 (useState, useEffect, custom hooks, common pitfalls). Provide code examples." |
| "What's new in Rust?" | "Rust 1.75 new features 2026 (async traits, impl Trait improvements, const generics, stabilized APIs). Include migration guide and code examples." |
| "PostgreSQL vs MySQL performance?" | "PostgreSQL vs MySQL performance comparison 2026 (query optimization, indexing strategies, concurrent writes, JSON handling, scaling patterns). Provide benchmark data and use case recommendations." |
| "How to handle errors in Go?" | "Go error handling patterns 2026 (error wrapping, custom errors, sentinel errors, panic vs error, testing error cases). Provide code examples and best practices comparison." |
| "Learn FastAPI basics" | "FastAPI tutorial 2026 (routing, dependency injection, async endpoints, request validation with Pydantic, OpenAPI documentation, testing). Include step-by-step implementation guide." |
Note: If user provides an already detailed query with version numbers and requirements, use it as-is.
Workflow
1. Receive user request 2. Optimize query using template above 3. Inform user: "Searching for: '[optimized query]'" 4. Execute search with --save --debug flags 5. Return results with inline citations [1][2][3]
Script Execution
CRITICAL: Always use the run.py wrapper. Direct script execution will fail.
Basic Search
python scripts/run.py search.py --query "Your search query"Recommended Usage
python scripts/run.py search.py --query "..." --save --debugThe run.py wrapper automatically:
- Creates
.venvon first run - Installs dependencies (patchright, beautifulsoup4, html-to-markdown)
- Activates virtual environment
- Executes search script
- Installs Google Chrome (not Chromium) for anti-detection
How It Works
1. Persistent Browser Context: Uses saved browser profile at ~/.cache/google-ai-mode-skill/chrome_profile to preserve cookies/session between searches 2. Eliminates CAPTCHAs: Persistent context means Google recognizes the browser → rarely triggers CAPTCHA 3. AI Content Detection: Waits for Google AI Overview to appear on page 4. Citation Extraction: Injects JavaScript to extract source links from AI response 5. Markdown Conversion: Converts HTML to markdown with inline citations [1][2][3] 6. Fast Results: Typical search completes in 5-7 seconds (no CAPTCHA)
CAPTCHA Handling
With persistent context, CAPTCHAs are rare. If encountered:
1. Detection: Multi-layer check (URL /sorry/index, page text, content length) 2. Automatic Handling: If CAPTCHA detected in headless mode → script returns CAPTCHA_REQUIRED error 3. Manual Solution: Re-run with --show-browser flag, solve CAPTCHA in browser, script continues automatically
Note: After CAPTCHA is solved once, persistent context preserves the session → future searches won't require CAPTCHA.
Output Format
Returns markdown with inline citations and source list. Example:
React 18 introduces concurrent features including Suspense for data fetching[1],
automatic batching for state updates[2], and transitions for non-urgent updates[3].
---
## Sources:
[1] React 18 Release Notes
https://react.dev/blog/2022/03/29/react-v18
[2] Automatic Batching Explained
https://github.com/reactwg/react-18/discussions/21
[3] Transitions API Documentation
https://react.dev/reference/react/useTransitionCommon Use Cases
Finding Library Documentation
python scripts/run.py search.py --query "Prisma ORM 2026 (schema definition, migrations, client API, relation queries, transactions). Include TypeScript examples." --save --debugGetting Coding Examples
python scripts/run.py search.py --query "WebSocket implementation Node.js 2026 (server setup, client connection, message handling, authentication, reconnection logic). Production-ready code examples." --saveTechnical Comparisons
python scripts/run.py search.py --query "GraphQL vs REST API 2026 (performance, caching, tooling, type safety, learning curve). Comparison table with use case recommendations." --saveBest Practices Research
python scripts/run.py search.py --query "Microservices security patterns 2026 (API gateway authentication, service mesh, mutual TLS, secret management, observability). Architecture diagrams and implementation guide." --save --debugTroubleshooting
| Issue | Solution |
|---|---|
ModuleNotFoundError | Use run.py wrapper, never execute scripts directly |
| CAPTCHA every time | First-time setup: solve CAPTCHA once with --show-browser, then persistent context preserves session |
| No AI overview found | Rephrase query with more specificity using optimization template |
| Browser fails to start | Verify internet connection and Chrome installation |
| Need detailed logs | Use --debug flag - log saved to logs/ folder |
| AI Mode not available | Your region/country doesn't support Google AI Mode. Use a proxy/VPN to access from supported regions (US, UK, Germany, etc.) |
Exit Codes:
0- Success1- General error2- CAPTCHA required (retry with--show-browser)3- Browser closed by user4- AI Mode not available in region (use proxy/VPN)130- User interrupted (Ctrl+C)
Best Practices
1. Always optimize queries - Specificity determines result quality 2. Use `--save --debug` for important searches - Preserves results and provides audit trail 3. Include version numbers for library/framework queries 4. Request structured output - Tables and comparisons improve usability 5. Solve CAPTCHA once - Persistent context eliminates future CAPTCHAs 6. Verify citations - Check provided sources for accuracy
# Virtual Environment
.venv/
venv/
env/
*.venv
# Skill Data (NEVER commit - contains auth and personal notebooks!)
data/
data/*
data/**/*
# Claude-specific
.claude/
*.claude
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
scripts/__pycache__/
scripts/*.pyc
# Environment
.env
*.env
.env.*
# Browser/Auth state (if accidentally placed outside data/)
browser_state/
auth/
auth_info.json
library.json
notebooks.json
state.json
cookies.json
# IDE
.vscode/
.idea/
*.swp
*.swo
*~
# OS
.DS_Store
.DS_Store?
._*
Thumbs.db
desktop.ini
ehthumbs.db
# Logs
*.log
logs/
*.debug
# Results
results/
*.md
!README.md
!SKILL.md
# Backups
*.backup
*.bak
*.tmp
*.temp
# Test artifacts
.coverage
htmlcov/
.pytest_cache/
.tox/
# Package artifacts
dist/
build/
*.egg-info/<div align="center">
Google AI Mode Skill
Supercharge Claude Code's Web Research with Google AI Mode
For: Claude Code CLI users only
Transform your LLM's online research capabilities by connecting Claude Code directly to Google's AI Mode—getting AI-synthesized answers from 100+ sources instead of scattered search results.
  
Why This Matters
Most built-in web research is mediocre. This skill gives Claude Code professional-grade research by tapping into Google's AI Mode—the same technology that synthesizes information from dozens of websites into one cited answer.
Example Use Cases:
"Next.js 15 App Router best practices 2026 with server components examples"
→ AI-synthesized coding guide with inline citations [1][2][3]
"Compare PostgreSQL vs MySQL JSON performance 2026, include benchmarks"
→ Technical comparison table with real-world data
"Find the latest EU AI regulations 2026 and their impact on startups"
→ Legal overview with official government sources
"Best noise-cancelling headphones under €300, compare Sony vs Bose"
→ Product comparison with reviews and specs
"Intermittent fasting protocols 2026, include recent scientific studies"
→ Health guide with medical research citationsResult: Research on ANY topic—coding, tech comparisons, regulations, product reviews, health, finance, travel. Curated answers with sources. Saves tokens. Superior to generic web search.
Installation • Quick Start • How It Works • MCP Alternative
</div>
---
📋 Last Updates (2026-01-08)
v2.0 - Multi-Language & Detection Overhaul
✅ 4-Stage Completion Detection - SVG thumbs-up → aria-label → text → 40s timeout ✅ Multi-Language Support - Works in DE/EN/NL/ES/FR/IT browser locales ✅ 87% Faster - Average 4s detection (was 30s+) ✅ AI Mode Availability Check - Detects region restrictions with proxy suggestion ✅ 17 Citation Selectors - Language-agnostic fallback chain ✅ 15 Cutoff Markers - Cleaner content extraction across languages
<details> <summary>📖 Show previous updates</summary>
v1.5 - Persistent browser profile, CAPTCHA elimination v1.0 - Initial release with basic Google AI Mode integration
</details>
---
⚠️ Important: Local Claude Code Only
This skill works ONLY with local [Claude Code](https://github.com/anthropics/claude-code) installations, NOT in the web UI.
The web UI runs skills in a sandbox without network access, which this skill requires for browser automation. You must use Claude Code locally on your machine.
---
What This Is
A Claude Code skill that connects your agent to Google AI Mode—Google's AI-powered search that synthesizes information from dozens of web sources into a single, cited answer.
Instead of Claude reading page after page, Google does the heavy lifting. Claude gets one clean, structured response with inline citations.
The advantage: Free, token-efficient research with grounded sources. No API keys needed.
---
How It Works
Claude asks a question
↓
Skill launches stealth browser
↓
Google AI Mode searches & synthesizes dozens of sources
↓
Skill extracts AI answer + citations
↓
Converts to clean Markdown with [1][2][3] references
↓
Claude receives final answerThe key difference:
Traditional web research:
- Claude searches Google → gets 10 links
- Claude reads 5-10 full pages → thousands of tokens consumed
- Claude synthesizes manually → risks missing details or hallucinating
- You pay for all those tokens
With this skill:
- Google AI Mode searches + synthesizes → one request
- Claude receives one clean, cited answer → minimal tokens
- Google's sources are preserved → verifiable, grounded
- It's free (uses public Google Search)
---
Why This Matters
Google AI Mode (the udm=50 parameter) makes Google search work like a research assistant. It:
- Reads and analyzes dozens of websites automatically
- Synthesizes findings into structured answers
- Cites every claim with source links
- Handles follow-up context across queries
Claude gets the benefits without doing the work—or burning the tokens.
---
Installation
The simplest installation ever:
# 1. Create skills directory (if it doesn't exist)
mkdir -p ~/.claude/skills
# 2. Clone this repository
cd ~/.claude/skills
git clone https://github.com/PleasePrompto/google-ai-mode-skill google-ai-mode
# 3. That's it! Open Claude Code and say:
"What are my skills?"When you first use the skill, it automatically:
- Creates an isolated Python environment (
.venv) - Installs all dependencies including Google Chrome
- Sets up browser automation with persistent profile
- Everything stays contained in the skill folder
Note: The setup uses real Chrome (not Chromium) for cross-platform reliability, consistent browser fingerprinting, and better anti-detection with Google services.
---
Quick Start
1. Check your skills
Say in Claude Code:
"What skills do I have?"Claude will list your available skills including Google AI Mode.
2. Start researching
"Search Google AI Mode for: Next.js 15 App Router best practices""What are the new features in Astro 4.0?""Research React Server Components"Claude will automatically use the skill to query Google AI Mode and return a clean, cited answer.
3. Get better results
Be specific with your queries:
Instead of: "React hooks" Try: "React hooks best practices 2026 (useState, useEffect, custom hooks). Include code examples."
Instead of: "PostgreSQL features" Try: "PostgreSQL 16 JSON features and performance improvements 2026"
---
First Run: CAPTCHA Handling
On your first query, Google may show a CAPTCHA to verify you're human. This is normal when the browser profile is created.
If Claude reports a CAPTCHA error: 1. Tell Claude: "Run that search with visible browser" 2. The browser will open visibly 3. Solve the CAPTCHA manually 4. The skill continues automatically 5. Next queries will work smoothly without CAPTCHAs
After the first CAPTCHA, searches typically run smoothly. The skill uses a persistent browser profile to eliminate future CAPTCHAs.
---
How the Skill Works
This is a Claude Code Skill—a local folder containing instructions and scripts that Claude Code can use when needed. Unlike the MCP server version, this runs directly in Claude Code without needing a separate server.
Key Differences from MCP Server
| Feature | This Skill | MCP Server |
|---|---|---|
| Protocol | Claude Skills | Model Context Protocol |
| Installation | Clone to ~/.claude/skills | claude mcp add ... |
| Compatibility | Claude Code only (local) | Claude Code, Codex, Cursor, Cline, etc. |
| Language | Python | TypeScript |
| Browser Profile | Persistent (eliminates CAPTCHAs) | Per-request context |
| Distribution | Git clone | npm package |
Architecture
~/.claude/skills/google-ai-mode/
├── SKILL.md # Instructions for Claude
├── scripts/ # Python automation scripts
│ ├── run.py # Universal venv wrapper
│ ├── search.py # Main search implementation
│ ├── browser_utils.py # Browser automation
│ └── config.py # Configuration
├── .venv/ # Isolated Python environment (auto-created)
└── results/ # Saved search results (optional)When Claude needs web research: 1. Loads the skill instructions from SKILL.md 2. Runs the Python search script via run.py wrapper 3. Opens browser with persistent profile (Chrome) 4. Extracts AI answer + citations from Google 5. Returns clean Markdown to Claude 6. Claude uses that knowledge to help with your task
---
Core Features
Source-Grounded Responses: Google AI Mode synthesizes information from dozens of sources with inline citations. Every claim is backed by a source link.
Direct Integration: No copy-paste between browser and editor. Claude queries and receives answers programmatically.
Persistent Browser Profile: After solving the first CAPTCHA (if any), the browser profile is saved. Future searches run smoothly without interruption.
Zero Configuration: Works out of the box. No API keys, no external services, no configuration files needed.
Self-Contained: Everything runs in the skill folder with an isolated Python environment. No global installations.
Token Efficient: One query returns one synthesized answer instead of Claude reading 5-10 full pages.
---
Troubleshooting
Skill not found:
# Make sure it's in the right location
ls ~/.claude/skills/google-ai-mode/
# Should show: SKILL.md, scripts/, requirements.txt, etc.Repeated CAPTCHAs:
If Google keeps showing CAPTCHAs:
- Tell Claude: "Use visible browser for this search"
- Add 10-30 second delays between searches
- Make sure the persistent profile isn't corrupted
Browser won't launch:
Clear the browser profile:
# Linux/macOS
rm -rf ~/.cache/google-ai-mode-skill/chrome_profile
# Windows
rmdir /s "%LOCALAPPDATA%\google-ai-mode-skill\chrome_profile"Dependencies issues:
# Manual reinstall if needed
cd ~/.claude/skills/google-ai-mode
rm -rf .venv
python -m venv .venv
source .venv/bin/activate # or .venv\Scripts\activate on Windows
pip install -r requirements.txt
python -m patchright install chromeWrong language results:
The skill forces English results. If you still get wrong languages, clear the browser profile (see above).
---
Tips for Better Results
Be specific with your queries:
Instead of: "React hooks" Try: "React hooks best practices 2026 (useState, useEffect, custom hooks, common pitfalls)"
Include version numbers:
Instead of: "Next.js features" Try: "Next.js 15 new features and breaking changes"
Request structured output:
"Compare PostgreSQL vs MySQL 2026 with a performance comparison table"
Ask for examples:
"Show me TypeScript discriminated union examples with type narrowing"
Use the query template:
[Technology/Topic] [Version] [Year] ([Aspect 1], [Aspect 2], [Aspect 3]). [Format request].---
Example Use Case
You need to implement OAuth2 in a framework you've never used before.
Traditional approach:
- Claude searches Google, gets 10 links
- Reads multiple documentation pages and blog posts
- Consumes thousands of tokens
- May miss important details or synthesize incorrectly
With this skill:
"Search Google AI Mode for: Hono OAuth2 implementation guide"- Google reads and synthesizes sources automatically
- Claude gets one structured answer with code examples and citations
- Minimal token usage
- Sources are linked for verification
Claude can then use this grounded information to write the actual implementation.
---
Technical Details
Core Technology:
- Patchright: Browser automation library (Playwright-based)
- Python 3.8+: Implementation language
- Real Chrome: Uses Google Chrome (not Chromium) for better reliability
- Persistent Context: Saves browser profile to eliminate CAPTCHAs
Dependencies:
patchright==1.57.2- Browser automationbeautifulsoup4==4.14.3- HTML parsinghtml-to-markdown==2.19.6- HTML conversion
Data Storage:
All data is stored locally within the skill directory:
~/.cache/google-ai-mode-skill/
└── chrome_profile/ - Persistent browser profile (cookies, session)---
Limitations
Skill-Specific:
- Local Claude Code only - Does not work in web UI (sandbox restrictions)
- Manual CAPTCHA solving - First query may require human verification
- Python dependency - Requires Python 3.8+ on your system
Google AI Mode:
- Rate limits - Frequent searches may trigger CAPTCHAs
- Public search only - No authentication required or supported
- Query quality matters - Vague queries may not trigger AI overviews
---
FAQ
Why doesn't this work in the Claude web UI? The web UI runs skills in a sandbox without network access. Browser automation requires network access to reach Google.
How is this different from the MCP server? This is a simpler, Python-based implementation that runs directly as a Claude Skill. The MCP server is more feature-rich and works with multiple tools (Codex, Cursor, etc.).
Can I use both this skill and the MCP server? Yes, but you probably don't need both. Use the skill for Claude Code, use the MCP server if you want multi-agent support (Cursor, Cline, etc.).
Is it free? Yes. The skill is open source, and it uses public Google Search. No API keys or subscriptions needed.
Is my data private? Everything runs locally on your machine. The browser profile stays on your computer. No credentials or external services required beyond Google Search.
What if the browser keeps crashing? Clear the browser profile (see Troubleshooting section) and try again.
---
Important Notes
CAPTCHA handling: Google may show a CAPTCHA on first use. Tell Claude to show the browser, solve it manually, and you're good to go for future searches.
Responsible use: This tool automates browser interactions with Google Search. Use it responsibly and be mindful of Google's Terms of Service. Add delays between heavy search sessions if needed.
Verification: While results come from Google's AI Mode with source citations, always verify critical information via the linked sources. This is a research tool, not a source of truth.
---
MCP Server Alternative
Using other code agents (Cursor, Codex, Cline, Windsurf)?
There's a full MCP server version of this tool that works with any MCP-compatible agent, not just Claude Code.
Check it out: google-ai-mode-mcp
The MCP version offers:
- Works with Claude Code, Codex, Cursor, Cline, Windsurf, Zed, etc.
- TypeScript implementation
- npm package distribution
- One-line installation:
claude mcp add google-ai-search npx google-ai-mode-mcp@latest
If you only use Claude Code, this skill is perfect. If you use multiple agents, consider the MCP server instead.
---
Contributing
Found an issue or want to contribute?
- Report bugs: GitHub Issues
- Pull requests: Welcome
- Contact: See parent repository
---
License
MIT License - see LICENSE file for details
---
Credits
This skill is inspired by the **Google AI Mode MCP Server** and provides an alternative implementation as a Claude Code Skill:
- Both use Patchright for browser automation (MCP uses TypeScript, Skill uses Python)
- Skill version runs directly in Claude Code without MCP protocol
- Optimized for Claude Code's skill architecture
If you need:
- Multi-agent support (Cursor, Cline, etc.) → Use the MCP Server
- Claude Code only → Use this skill
- npm distribution → Use the MCP Server
- Git clone simplicity → Use this skill
---
The Bottom Line
Without this skill: Claude searches Google → Gets links → Reads 5-10 pages → Thousands of tokens → Potential hallucinations
With this skill: Claude queries Google AI Mode → Gets one synthesized answer with citations → Minimal tokens → Grounded results
Stop burning tokens on web research. Start getting accurate, cited answers directly in Claude Code.
# Get started in 30 seconds
cd ~/.claude/skills
git clone https://github.com/PleasePrompto/google-ai-mode-skill google-ai-mode
# Open Claude Code: "What are my skills?"---
<div align="center">
Built as a Claude Code Skill adaptation of the Google AI Mode MCP Server
For free, token-efficient web research directly in Claude Code
</div>
# Google AI Mode Skill Dependencies
# Installed in local .venv via run.py
# Core browser automation with anti-detection
patchright==1.57.2
# HTML parsing for content extraction
beautifulsoup4==4.14.3
# HTML to Markdown conversion
# Primary option (Rust-based, fast)
html-to-markdown==2.19.6
# Fallback options (will be used if primary fails)
# markdownify>=0.11.6
# html2text>=2020.1.16
"""
Browser Utilities for Google AI Mode Skill
Uses persistent context to avoid CAPTCHAs
"""
import time
import random
from typing import Optional
from patchright.sync_api import Playwright, Browser, BrowserContext, Page
from config import BROWSER_ARGS, USER_AGENT, BROWSER_PROFILE_DIR, LOCALE, EXTRA_HTTP_HEADERS
class BrowserFactory:
"""Factory for creating configured browser instances"""
@staticmethod
def launch_persistent_context(playwright: Playwright, headless: bool = True) -> BrowserContext:
"""
Launch browser with PERSISTENT CONTEXT - keeps cookies/session!
This dramatically reduces CAPTCHA occurrences.
Sets English as preferred language (but multi-language selectors handle any locale).
"""
import json
# Step 1: Set Local State (profile-wide settings)
local_state_file = BROWSER_PROFILE_DIR / "Local State"
local_state = {}
if local_state_file.exists():
try:
with open(local_state_file, 'r', encoding='utf-8') as f:
local_state = json.load(f)
except:
local_state = {}
# Force English in Local State
local_state.update({
"intl": {
"app_locale": "en", # CRITICAL: Chrome UI language
"accept_languages": "en-US,en"
}
})
with open(local_state_file, 'w', encoding='utf-8') as f:
json.dump(local_state, f, indent=2)
# Step 2: Set Default/Preferences (per-profile settings)
prefs_dir = BROWSER_PROFILE_DIR / "Default"
prefs_dir.mkdir(parents=True, exist_ok=True)
prefs_file = prefs_dir / "Preferences"
prefs = {}
if prefs_file.exists():
# Load existing preferences to preserve cookies/session
try:
with open(prefs_file, 'r', encoding='utf-8') as f:
prefs = json.load(f)
except:
prefs = {}
# FORCE English language settings
prefs.update({
"intl": {
"accept_languages": "en-US,en",
"selected_languages": "en-US,en",
"app_locale": "en" # Redundant but ensures consistency
},
"translate": {
"enabled": False # Disable auto-translate
},
"webkit": {
"webprefs": {
"default_charset": "utf-8"
}
}
})
# Write preferences atomically
with open(prefs_file, 'w', encoding='utf-8') as f:
json.dump(prefs, f, indent=2)
# NOW launch browser (will read our forced preferences)
return playwright.chromium.launch_persistent_context(
str(BROWSER_PROFILE_DIR), # Persistent profile directory
channel="chrome", # Use real Chrome for better anti-detection
headless=headless,
user_agent=USER_AGENT,
locale=LOCALE, # Force English locale
extra_http_headers=EXTRA_HTTP_HEADERS, # Force English language headers
args=BROWSER_ARGS,
ignore_default_args=["--enable-automation"],
)
@staticmethod
def launch_browser(playwright: Playwright, headless: bool = True) -> Browser:
"""
Launch browser with anti-detection features.
DEPRECATED: Use launch_persistent_context instead to avoid CAPTCHAs!
"""
return playwright.chromium.launch(
channel="chrome", # Use real Chrome for better anti-detection
headless=headless,
args=BROWSER_ARGS
)
class StealthUtils:
"""Human-like interaction utilities"""
@staticmethod
def random_delay(min_ms: int = 100, max_ms: int = 500):
"""Add random delay"""
time.sleep(random.uniform(min_ms / 1000, max_ms / 1000))
@staticmethod
def human_type(page: Page, selector: str, text: str, wpm_min: int = 320, wpm_max: int = 480):
"""Type with human-like speed"""
element = page.query_selector(selector)
if not element:
# Try waiting if not immediately found
try:
element = page.wait_for_selector(selector, timeout=2000)
except:
pass
if not element:
print(f"⚠️ Element not found for typing: {selector}")
return
# Click to focus
element.click()
# Type
for char in text:
element.type(char, delay=random.uniform(25, 75))
if random.random() < 0.05:
time.sleep(random.uniform(0.15, 0.4))
@staticmethod
def realistic_click(page: Page, selector: str):
"""Click with realistic movement"""
element = page.query_selector(selector)
if not element:
return
# Optional: Move mouse to element (simplified)
box = element.bounding_box()
if box:
x = box['x'] + box['width'] / 2
y = box['y'] + box['height'] / 2
page.mouse.move(x, y, steps=5)
StealthUtils.random_delay(100, 300)
element.click()
StealthUtils.random_delay(100, 300)
"""
Configuration for Google AI Mode Skill
Minimal config - no auth, no persistence needed
"""
from pathlib import Path
import os
import sys
# Paths
SKILL_DIR = Path(__file__).parent.parent
RESULTS_DIR = SKILL_DIR / "results"
LOGS_DIR = SKILL_DIR / "logs"
# Browser Profile - persistent context to avoid CAPTCHAs!
# Store in user's home directory for persistence across sessions
# Platform-specific cache directories:
# - Windows: %LOCALAPPDATA%\google-ai-mode-skill\chrome_profile
# - macOS: ~/Library/Caches/google-ai-mode-skill/chrome_profile
# - Linux: ~/.cache/google-ai-mode-skill/chrome_profile
if sys.platform == "win32":
# Windows: Use AppData\Local
BROWSER_PROFILE_DIR = Path(os.getenv("LOCALAPPDATA", Path.home() / "AppData" / "Local")) / "google-ai-mode-skill" / "chrome_profile"
elif sys.platform == "darwin":
# macOS: Use ~/Library/Caches
BROWSER_PROFILE_DIR = Path.home() / "Library" / "Caches" / "google-ai-mode-skill" / "chrome_profile"
else:
# Linux/Unix: Use ~/.cache
BROWSER_PROFILE_DIR = Path.home() / ".cache" / "google-ai-mode-skill" / "chrome_profile"
BROWSER_PROFILE_DIR.mkdir(parents=True, exist_ok=True)
# Browser Configuration
BROWSER_ARGS = [
'--disable-blink-features=AutomationControlled',
'--disable-dev-shm-usage',
'--no-sandbox',
'--no-first-run',
'--no-default-browser-check',
'--lang=en', # CRITICAL: Must be 'en' not 'en-US' for UI language!
'--disable-translate', # Disable auto-translate popup
]
USER_AGENT = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36'
# Locale settings for consistent language
LOCALE = "en-US"
EXTRA_HTTP_HEADERS = {
"Accept-Language": "en-US,en;q=0.9"
}
# Timeouts
PAGE_LOAD_TIMEOUT = 45000 # 45 seconds
AI_RESPONSE_TIMEOUT = 30 # 30 seconds
"""
Logging System for Google AI Mode Skill
Provides comprehensive debug logging with file and console output
"""
import logging
import sys
from pathlib import Path
from datetime import datetime
class SkillLogger:
"""Zentrales Logging-System für Google AI Mode Skill"""
def __init__(self, debug: bool = False):
self.debug_enabled = debug
self.logger = None
self.log_file = None
if debug:
self._setup_logger()
def _setup_logger(self):
"""Konfiguriert Logger mit File und Console Handlers"""
# Log-Ordner erstellen
log_dir = Path(__file__).parent.parent / "logs"
log_dir.mkdir(exist_ok=True)
# Timestamp-basierter Log-Filename
timestamp = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
self.log_file = log_dir / f"search_{timestamp}.log"
# Logger konfigurieren
self.logger = logging.getLogger("GoogleAIMode")
self.logger.setLevel(logging.DEBUG)
# Clear existing handlers (avoid duplicates)
self.logger.handlers.clear()
# File Handler - speichert ALLE Debug-Infos
fh = logging.FileHandler(self.log_file, encoding='utf-8')
fh.setLevel(logging.DEBUG)
# Console Handler - nur wichtige Meldungen
ch = logging.StreamHandler(sys.stdout)
ch.setLevel(logging.INFO)
# Format mit Timestamp
formatter = logging.Formatter(
'%(asctime)s - %(levelname)s - %(message)s',
datefmt='%H:%M:%S'
)
fh.setFormatter(formatter)
ch.setFormatter(formatter)
self.logger.addHandler(fh)
self.logger.addHandler(ch)
self.logger.info(f"Debug logging enabled - Log file: {self.log_file}")
def debug(self, msg):
"""Debug-level logging (nur in File, nicht in Console)"""
if self.debug_enabled and self.logger:
self.logger.debug(msg)
def info(self, msg):
"""Info-level logging (in File und Console)"""
if self.debug_enabled and self.logger:
self.logger.info(msg)
def warning(self, msg):
"""Warning-level logging (in File und Console)"""
if self.debug_enabled and self.logger:
self.logger.warning(msg)
def error(self, msg):
"""Error-level logging (in File und Console)"""
if self.debug_enabled and self.logger:
self.logger.error(msg)
def exception(self, msg):
"""Exception-level logging mit Traceback"""
if self.debug_enabled and self.logger:
self.logger.exception(msg)
# Dummy-Logger für non-debug Modus
class DummyLogger:
"""Dummy-Logger der nichts tut (für non-debug mode)"""
def __init__(self):
self.debug_enabled = False
self.log_file = None
def debug(self, msg):
pass
def info(self, msg):
pass
def warning(self, msg):
pass
def error(self, msg):
pass
def exception(self, msg):
pass
def get_logger(debug: bool = False):
"""Factory function für Logger"""
if debug:
return SkillLogger(debug=True)
else:
return DummyLogger()
#!/usr/bin/env python3
"""
Reset Browser Profile - Fix Language Issues
Deletes the persistent browser profile to force fresh creation with English settings.
Use this if Google still shows German interface after language fixes.
Usage:
python scripts/run.py reset_profile.py
"""
import shutil
import sys
from pathlib import Path
# Add parent directory to path
sys.path.insert(0, str(Path(__file__).parent))
from config import BROWSER_PROFILE_DIR
def main():
print("=" * 60)
print("BROWSER PROFILE RESET")
print("=" * 60)
print()
if BROWSER_PROFILE_DIR.exists():
print(f"📁 Profile location: {BROWSER_PROFILE_DIR}")
print(f"⚠️ This will delete all browser data:")
print(f" - Cached language settings (fixes German interface)")
print(f" - Cookies and session (may trigger CAPTCHA on next search)")
print(f" - Login state")
print()
response = input("Continue? (y/N): ").strip().lower()
if response != 'y':
print("❌ Cancelled.")
return 1
print()
print(f"🗑️ Deleting profile...")
shutil.rmtree(BROWSER_PROFILE_DIR)
print(f"✅ Profile deleted!")
print()
print(f"📝 Next steps:")
print(f" 1. Run a search with --show-browser:")
print(f" python scripts/run.py search.py --query 'test' --show-browser")
print(f" 2. Verify Google shows ENGLISH interface")
print(f" 3. If CAPTCHA appears, solve it once")
print(f" 4. Profile will be recreated with English settings")
print()
return 0
else:
print(f"ℹ️ No profile found at: {BROWSER_PROFILE_DIR}")
print(f" Profile will be created on first search.")
print()
return 0
if __name__ == "__main__":
sys.exit(main())
#!/usr/bin/env python3
"""
Universal runner for NotebookLM skill scripts
Ensures all scripts run with the correct virtual environment
"""
import os
import sys
import subprocess
from pathlib import Path
def get_venv_python():
"""Get the virtual environment Python executable"""
skill_dir = Path(__file__).parent.parent
venv_dir = skill_dir / ".venv"
if os.name == 'nt': # Windows
venv_python = venv_dir / "Scripts" / "python.exe"
else: # Unix/Linux/Mac
venv_python = venv_dir / "bin" / "python"
return venv_python
def ensure_venv():
"""Ensure virtual environment exists"""
skill_dir = Path(__file__).parent.parent
venv_dir = skill_dir / ".venv"
setup_script = skill_dir / "scripts" / "setup_environment.py"
# Check if venv exists
if not venv_dir.exists():
print("🔧 First-time setup: Creating virtual environment...")
print(" This may take a minute...")
# Run setup with system Python
result = subprocess.run([sys.executable, str(setup_script)])
if result.returncode != 0:
print("❌ Failed to set up environment")
sys.exit(1)
print("✅ Environment ready!")
return get_venv_python()
def main():
"""Main runner"""
if len(sys.argv) < 2:
print("Usage: python run.py <script_name> [args...]")
print("\nAvailable scripts:")
print(" search.py - Query Google AI Mode for web research")
print("\nExample:")
print(' python run.py search.py --query "React hooks 2026" --save --debug')
sys.exit(1)
script_name = sys.argv[1]
script_args = sys.argv[2:]
# Handle both "scripts/script.py" and "script.py" formats
if script_name.startswith('scripts/'):
# Remove the scripts/ prefix if provided
script_name = script_name[8:] # len('scripts/') = 8
# Ensure .py extension
if not script_name.endswith('.py'):
script_name += '.py'
# Get script path
skill_dir = Path(__file__).parent.parent
script_path = skill_dir / "scripts" / script_name
if not script_path.exists():
print(f"❌ Script not found: {script_name}")
print(f" Working directory: {Path.cwd()}")
print(f" Skill directory: {skill_dir}")
print(f" Looked for: {script_path}")
sys.exit(1)
# Ensure venv exists and get Python executable
venv_python = ensure_venv()
# Build command
cmd = [str(venv_python), str(script_path)] + script_args
# Run the script
try:
result = subprocess.run(cmd)
sys.exit(result.returncode)
except KeyboardInterrupt:
print("\n⚠️ Interrupted by user")
sys.exit(130)
except Exception as e:
print(f"❌ Error: {e}")
sys.exit(1)
if __name__ == "__main__":
main()#!/usr/bin/env python3
"""
-------------------------------------------------------------------------------
Google AI Mode Search
-------------------------------------------------------------------------------
Searches Google's AI mode (udm=50) and extracts AI-generated overviews with
citations for use in Claude Code.
Features:
- Automatic captcha detection
- Graceful error handling
- Citation extraction with sources
- HTML to Markdown conversion
Usage:
python scripts/run.py search.py --query "Your search query"
python scripts/run.py search.py --query "..." --show-browser
"""
import sys
import os
import time
import json
import argparse
import re
from pathlib import Path
from typing import List, Dict, Optional, Any
from datetime import datetime
# Third-party imports
from patchright.sync_api import sync_playwright, Page
from bs4 import BeautifulSoup
# Local imports
from browser_utils import BrowserFactory
from config import USER_AGENT, PAGE_LOAD_TIMEOUT, AI_RESPONSE_TIMEOUT, RESULTS_DIR, BROWSER_PROFILE_DIR
from logger import get_logger
try:
from html_to_markdown import convert, ConversionOptions
except ImportError:
# Fallback if html-to-markdown not available
print("⚠️ Warning: html-to-markdown not found, trying markdownify...")
try:
from markdownify import markdownify as md
def convert(html, options=None):
return md(html)
ConversionOptions = None
except ImportError:
print("⚠️ Warning: markdownify not found either, using html2text...")
try:
import html2text
h = html2text.HTML2Text()
h.body_width = 0
def convert(html, options=None):
return h.handle(html)
ConversionOptions = None
except ImportError:
print("❌ Error: No HTML to Markdown converter found!")
print(" Install one of: html-to-markdown, markdownify, or html2text")
sys.exit(1)
# =============================================================================
# MULTI-LANGUAGE SELECTORS (DE/EN/NL support)
# =============================================================================
# Citation button selectors - ALL languages
CITATION_SELECTORS = [
'[aria-label="View related links"]', # English
'[aria-label*="Related links"]', # English partial
'[aria-label="Zugehörige Links anzeigen"]', # German
'[aria-label*="Zugehörige Links"]', # German partial
'[aria-label*="Gerelateerde links"]', # Dutch partial
'button[aria-label*="links" i]', # Generic case-insensitive
]
# AI Completion Detection - Multi-language
AI_COMPLETION_BUTTON = '[aria-label*="feedback" i]' # Language-independent
AI_COMPLETION_TIMEOUT = 15000 # 15 seconds (SERPO proven)
# Text-based completion indicators (fallback)
AI_COMPLETION_TEXT_INDICATORS = [
# English
'AI-generated', 'AI Overview', 'Generative AI is experimental',
# German
'KI-Antworten', 'KI-generiert', 'Generative KI',
# Dutch
'AI-gegenereerd', 'AI-overzicht',
# Spanish
'Las respuestas de la IA', 'Resumen de IA', 'Información general de IA',
# French
'Réponses IA', "Aperçu de l'IA", "Vue d'ensemble de l'IA",
# Italian
'Risposte IA', "Panoramica IA", "Panoramica dell'IA",
]
# Disclaimer cutoff markers (remove everything after these)
CUTOFF_MARKERS = [
# German
'KI-Antworten können Fehler enthalten',
'Öffentlicher Link wird erstellt',
# English
'AI-generated answers may contain mistakes',
'AI can make mistakes',
'Generative AI is experimental',
# Dutch
'AI-reacties kunnen fouten bevatten',
# Spanish
'Las respuestas de la IA pueden contener errores',
'pueden contener errores',
'Más información',
# French
"Les réponses de l'IA peuvent contenir des erreurs",
'peuvent contenir des erreurs',
'Plus d\'informations',
# Italian
"Le risposte dell'IA possono contenere errori",
'possono contenere errori',
'Ulteriori informazioni',
]
# AI Mode not available indicators (region/language restrictions)
AI_MODE_NOT_AVAILABLE = [
# French
"Le Mode IA n'est pas disponible dans votre pays ou votre langue",
"Mode IA n'est pas disponible",
"Découvrez le Mode IA",
# English
"AI Mode is not available in your country or language",
"AI Mode isn't available",
# German
"Der KI-Modus ist in Ihrem Land oder Ihrer Sprache nicht verfügbar",
"KI-Modus ist nicht verfügbar",
# Spanish
"El modo de IA no está disponible en tu país o idioma",
# Italian
"La modalità IA non è disponibile nel tuo Paese o nella tua lingua",
# Dutch
"AI-modus is niet beschikbaar in uw land of taal",
]
# =============================================================================
# JAVASCRIPT INJECTION CODE
# =============================================================================
DOM_INJECTION_SCRIPT = '''
async () => {
// Helper: Prüft ob ein Element visuell für den User sichtbar ist
function isVisible(el) {
if (!el) return false;
const style = window.getComputedStyle(el);
const rect = el.getBoundingClientRect();
return style.display !== 'none' &&
style.visibility !== 'hidden' &&
style.opacity !== '0' &&
el.offsetParent !== null &&
rect.width > 0 &&
rect.height > 0;
}
// Haupt-Container der AI Overview finden
const mainCol = document.querySelector('[data-container-id="main-col"]');
if (!mainCol) return { error: 'main-col not found (AI Overview missing?)' };
// SERPO OPTIMIZATION: Expand "Show more" buttons first
try {
const showMoreBtns = Array.from(mainCol.querySelectorAll('[aria-expanded="false"]'));
for (const btn of showMoreBtns) {
if (isVisible(btn) && (btn.innerText.includes('Show more') ||
btn.innerText.includes('Mehr anzeigen') ||
btn.innerText.includes('Meer weergeven'))) {
btn.click();
await new Promise(r => setTimeout(r, 200));
}
}
} catch (e) {
console.warn('Show more expansion failed', e);
}
// MULTI-LANGUAGE: Try citation selectors in order (injected from Python)
const selectors = %CITATION_SELECTORS%;
let buttons = [];
for (const selector of selectors) {
buttons = Array.from(mainCol.querySelectorAll(selector));
if (buttons.filter(isVisible).length > 0) {
console.log(`Found ${buttons.length} citation buttons with: ${selector}`);
break;
}
}
const allCitations = [];
let markerIndex = 0;
for (const btn of buttons) {
// Ignoriere unsichtbare "Geister"-Buttons im DOM
if (!isVisible(btn)) continue;
// 1. Marker [CITE-N] visuell einfügen (SERPO: wrapped in <code> tag!)
const markerId = markerIndex++;
const marker = document.createElement('span');
marker.className = 'citation-marker';
marker.innerHTML = `<code>[CITE-${markerId}]</code>`;
// Marker hinter dem Button platzieren
if (btn.nextSibling) {
btn.parentNode.insertBefore(marker, btn.nextSibling);
} else {
btn.parentNode.appendChild(marker);
}
// 2. Button klicken, um Quellen in der Seitenleiste zu laden
try {
btn.scrollIntoView({ behavior: 'instant', block: 'center' });
// Zähle sichtbare Links VOR dem Klick
const countVisibleLinks = () => {
const rhsCol = document.querySelector('[data-container-id="rhs-col"]');
if (!rhsCol) return 0;
return Array.from(rhsCol.querySelectorAll('a[href]')).filter(isVisible).length;
};
const beforeCount = countVisibleLinks();
btn.click();
// Smart Wait: Warte kurz, ob sich Links ändern (max 300ms)
const startTime = Date.now();
while (Date.now() - startTime < 300) {
await new Promise(r => setTimeout(r, 10));
if (countVisibleLinks() !== beforeCount) break;
}
// Kurzer Puffer für Animationen
await new Promise(r => setTimeout(r, 50));
} catch (e) {
console.warn('Click failed', e);
}
// 3. Quellen aus der Seitenleiste (rhs-col) extrahieren
const sources = [];
const seen = new Set();
const rhsCol = document.querySelector('[data-container-id="rhs-col"]');
if (rhsCol) {
const links = Array.from(rhsCol.querySelectorAll('a[href]'));
for (const link of links) {
if (!isVisible(link)) continue;
const url = link.href;
const title = link.innerText.trim() || link.getAttribute('aria-label') || '';
// Google-Interne Domains filtern
const skipDomains = ['google.com', 'google.de', 'gstatic.com', 'support.google.com'];
if (url && url.startsWith('http') && !skipDomains.some(d => url.includes(d)) && !seen.has(url)) {
seen.add(url);
sources.push({
title: title,
url: url,
source: new URL(url).hostname
});
}
}
}
allCitations.push({ marker_id: markerId, sources: sources });
}
// Rückgabe: Das modifizierte HTML (mit Markern) + die extrahierten Quellen
return {
html: mainCol.innerHTML,
citations: allCitations
};
}
'''
# =============================================================================
# CAPTCHA DETECTION (3-Layer Strategy)
# =============================================================================
def detect_captcha(page: Page) -> bool:
"""
Erkennt ob Google ein Captcha zeigt (3-Layer Detection)
Layer 1: URL contains /sorry/index
Layer 2: Body text contains "unusual traffic"
Layer 3: Page content is very short (< 600 chars)
Returns True if ANY layer detects CAPTCHA
"""
# LAYER 1: URL-Check (Most reliable!)
# Google's CAPTCHA pages always redirect to /sorry/index
try:
current_url = page.url
if '/sorry/index' in current_url or 'google.com/sorry' in current_url:
print(" 🔍 CAPTCHA detected (Layer 1: URL contains /sorry/index)")
return True
except:
pass
# LAYER 2: Text-Check
# CAPTCHA pages contain "unusual traffic" text
try:
body = page.inner_text('body')
body_lower = body.lower()
unusual_traffic_indicators = [
'unusual traffic',
'ungewöhnlichen datenverkehr',
'unsere systeme haben',
'our systems have detected'
]
for indicator in unusual_traffic_indicators:
if indicator in body_lower:
print(f" 🔍 CAPTCHA detected (Layer 2: Text contains '{indicator}')")
return True
except:
pass
# LAYER 3: Length-Check
# CAPTCHA pages are very short (< 600 chars)
# Real AI Overview pages are much longer (usually > 2000 chars)
try:
body = page.inner_text('body')
body_length = len(body.strip())
if body_length < 600:
# Double-check with text to avoid false positives
body_lower = body.lower()
if 'captcha' in body_lower or 'unusual' in body_lower or 'über diese seite' in body_lower:
print(f" 🔍 CAPTCHA detected (Layer 3: Page too short - {body_length} chars)")
return True
except:
pass
# LEGACY: Element-based detection (backup)
# Less reliable but catches some edge cases
captcha_selectors = [
'div#recaptcha',
'iframe[src*="recaptcha"]',
'[id*="captcha"]',
]
for selector in captcha_selectors:
try:
if page.query_selector(selector):
print(f" 🔍 CAPTCHA detected (Legacy: Element {selector} found)")
return True
except:
pass
return False
# =============================================================================
# MAIN SCRAPER CLASS
# =============================================================================
class GoogleAIScraper:
def __init__(self, headless: bool = True, logger=None):
self.headless = headless
self.logger = logger if logger else get_logger(debug=False)
self.pw = None
self.ctx = None # Persistent context (no separate browser object needed)
self.page = None
def start(self):
"""Startet den Browser mit PERSISTENT CONTEXT"""
self.logger.debug(f"Starting browser with persistent context (headless={self.headless})...")
self.logger.debug(f"Profile directory: {BROWSER_PROFILE_DIR}")
self.pw = sync_playwright().start()
factory = BrowserFactory()
# Use persistent context - keeps cookies/session between runs!
self.ctx = factory.launch_persistent_context(self.pw, headless=self.headless)
self.logger.info("✅ Persistent context launched (cookies preserved!)")
self.page = self.ctx.new_page()
self.logger.debug("Browser page created")
def stop(self):
"""Beendet den Browser"""
self.logger.debug("Cleaning up browser resources...")
try:
if self.page:
self.page.close()
self.logger.debug("Page closed")
except Exception as e:
self.logger.debug(f"Error closing page: {e}")
try:
if self.ctx:
self.ctx.close()
self.logger.debug("Persistent context closed (profile saved)")
except Exception as e:
self.logger.debug(f"Error closing context: {e}")
try:
if self.pw:
self.pw.stop()
self.logger.debug("Playwright stopped")
except Exception as e:
self.logger.debug(f"Error stopping playwright: {e}")
def _clean_html_pre_processing(self, html: str) -> str:
"""Entfernt störende Links aus Code-Blöcken vor der Markdown-Konvertierung"""
soup = BeautifulSoup(html, 'html.parser')
# <a> Tags in <pre> und <code> entfernen
for block in soup.find_all(['pre', 'code']):
for link in block.find_all('a', href=True):
# Ersetze Link durch reinen Text (URL)
link.replace_with(link.get('href', ''))
return str(soup)
def _extract_sidebar_fallback(self) -> List[Dict]:
"""
SERPO-inspired fallback: Extract sources from sidebar when DOM injection fails
Triggered when:
- No citation buttons found
- DOM injection returns empty citations
- JavaScript errors
Returns:
List[Dict]: [{'title': str, 'url': str, 'source': str}, ...]
"""
try:
self.logger.debug("Sidebar fallback: extracting sources...")
# Get sidebar container
sidebar = self.page.query_selector('[data-container-id="rhs-col"]')
if not sidebar:
self.logger.debug("Sidebar not found")
return []
# Extract all links
links = sidebar.query_selector_all('a[href]')
sources = []
seen_urls = set()
# Filter domains (same as DOM injection)
skip_domains = ['google.com', 'google.de', 'gstatic.com', 'support.google.com']
for link in links:
try:
url = link.get_attribute('href')
title = link.inner_text().strip() or link.get_attribute('aria-label') or ''
# Skip invalid/duplicate/Google URLs
if not url or not url.startswith('http') or url in seen_urls:
continue
if any(domain in url for domain in skip_domains):
continue
# Parse domain
from urllib.parse import urlparse
domain = urlparse(url).hostname or ''
sources.append({
'title': title,
'url': url,
'source': domain
})
seen_urls.add(url)
except Exception as e:
self.logger.debug(f"Link parse error: {e}")
continue
self.logger.info(f"Sidebar fallback: {len(sources)} sources")
return sources
except Exception as e:
self.logger.error(f"Sidebar fallback failed: {e}")
return []
def _embed_citations(self, markdown: str, citations: List[Dict]) -> tuple:
"""Ersetzt [CITE-N] Marker durch [1][2] Fußnoten"""
modified_md = markdown
citation_sources = []
# Sortieren (höchste ID zuerst), damit beim Ersetzen Indizes stimmen
citations_sorted = sorted(citations, key=lambda c: c.get('marker_id', 999), reverse=True)
for citation in citations_sorted:
marker_id = citation.get('marker_id')
marker = f'[CITE-{marker_id}]'
sources = citation.get('sources', [])
if sources:
start_idx = len(citation_sources)
# Erzeuge Fußnoten-String: [1][2]
footnotes = ''.join(f'[{start_idx + i + 1}]' for i in range(len(sources)))
# Ersetze den Marker im Text
if marker in modified_md:
modified_md = modified_md.replace(marker, footnotes, 1)
citation_sources.extend(sources)
# Entferne übrig gebliebene Marker (falls keine Sources gefunden wurden)
modified_md = re.sub(r'\[CITE-\d+\]', '', modified_md)
return modified_md, citation_sources
def scrape(self, query: str) -> Dict[str, Any]:
"""Führt den kompletten Scraping-Prozess durch"""
if not self.page:
raise RuntimeError("Browser not started. Call start() first.")
url = f"https://www.google.com/search?udm=50&q={query.replace(' ', '+')}"
print(f" 🌐 Loading Query: {query[:50]}...")
self.logger.debug(f"Navigating to: {url}")
try:
self.page.goto(url, wait_until="domcontentloaded", timeout=PAGE_LOAD_TIMEOUT)
self.logger.debug("Page loaded successfully")
except Exception as e:
# Check for browser closed error
self.logger.error(f"Page load failed: {e}")
if "browser has been closed" in str(e).lower() or "target closed" in str(e).lower():
return {
"success": False,
"error": "BROWSER_CLOSED_BY_USER",
"message": "Browser wurde vom User geschlossen"
}
return {"success": False, "error": f"Page load timeout: {e}"}
# CAPTCHA CHECK (nach page load)
print(f" 🔍 Checking for CAPTCHA...")
self.logger.debug("Checking for CAPTCHA...")
if detect_captcha(self.page):
self.logger.warning("CAPTCHA detected")
if self.headless:
# Headless mode: Error zurückgeben
self.logger.info("Running in headless mode - returning CAPTCHA error")
return {
"success": False,
"error": "CAPTCHA_REQUIRED",
"message": "Google requires CAPTCHA verification. Please run again with --show-browser flag."
}
else:
# Visible mode: Informiere User, aber KEIN Polling!
# Der "Waiting for AI content" Loop unten wartet automatisch
print("⚠️ CAPTCHA DETECTED - Browser bleibt offen")
print(" Bitte lösen Sie das Captcha im Browser")
print(" Script wartet automatisch auf AI-Antwort...")
self.logger.info("CAPTCHA detected - waiting for user to solve and AI content to appear...")
else:
self.logger.debug("No CAPTCHA detected, proceeding...")
# CHECK FOR AI MODE AVAILABILITY (region/language restrictions)
print(f" 🌍 Checking AI Mode availability...")
self.logger.debug("Checking if AI Mode is available in this region/language...")
try:
body_text = self.page.inner_text('body')
if any(indicator in body_text for indicator in AI_MODE_NOT_AVAILABLE):
self.logger.error("AI Mode not available in this region/language")
print(f" ❌ AI Mode not available in your country/language")
return {
"success": False,
"error": "AI_MODE_NOT_AVAILABLE",
"message": "Google AI Mode is not available in your country or language. Please use a proxy/VPN to access from a supported region (e.g., US, UK, Germany).",
"suggestion": "Try using a proxy/VPN and ensure browser locale is set to a supported language."
}
else:
self.logger.debug("AI Mode available, proceeding...")
except Exception as e:
self.logger.debug(f"Could not check AI Mode availability: {e}")
# Proceed anyway - don't block on this check
# HYBRID AI COMPLETION DETECTION (SERPO method + Multi-language fallback)
print(f" ⏳ Waiting for AI completion...")
self.logger.debug("Starting hybrid completion detection...")
ai_ready = False
# OVERALL TIMEOUT: 40 seconds total, then proceed anyway
overall_deadline = time.time() + 40
# PRIMARY: Button-based detection (DUAL METHOD - ultra-robust!)
# Method 1: SVG-based detection (100% reliable, language-independent!)
remaining_time = int((overall_deadline - time.time()) * 1000)
if remaining_time > 0 and not ai_ready:
try:
self.logger.debug("Method 1: Attempting SVG thumbs-up icon detection...")
svg_selector = 'button svg[viewBox="3 3 18 18"]'
self.page.wait_for_selector(
svg_selector,
timeout=min(AI_COMPLETION_TIMEOUT, remaining_time),
state='visible'
)
ai_ready = True
self.logger.info("✅ Thumbs UP SVG detected!")
print(f" ✅ AI complete (Thumbs UP SVG detected!)")
except Exception as svg_error:
# Method 2: aria-label detection (fallback)
self.logger.debug(f"Method 1 failed: {svg_error}")
remaining_time = int((overall_deadline - time.time()) * 1000)
if remaining_time > 0 and not ai_ready:
try:
self.logger.debug(f"Method 2: Attempting aria-label detection: {AI_COMPLETION_BUTTON}")
self.page.wait_for_selector(
AI_COMPLETION_BUTTON,
timeout=min(AI_COMPLETION_TIMEOUT, remaining_time),
state='visible'
)
ai_ready = True
self.logger.info("✅ AI complete via aria-label button")
print(f" ✅ AI complete (button aria-label detected)")
except Exception as aria_error:
# Method 3: Text-based detection (multi-language fallback)
self.logger.debug(f"Method 2 failed: {svg_error}")
self.logger.debug("Both button methods failed, trying text detection...")
print(f" ⏳ Button not found, trying text detection (multi-lang)...")
# Text fallback: Poll until overall deadline
while time.time() < overall_deadline and not ai_ready:
try:
body = self.page.inner_text('body')
if any(indicator in body for indicator in AI_COMPLETION_TEXT_INDICATORS):
ai_ready = True
self.logger.info(f"✅ AI complete via text")
print(f" ✅ AI complete (text detected)")
break
except Exception as e:
if "browser has been closed" in str(e).lower() or "target closed" in str(e).lower():
self.logger.error("Browser closed while waiting for AI content")
return {
"success": False,
"error": "BROWSER_CLOSED_BY_USER",
"message": "Browser wurde vom User geschlossen"
}
time.sleep(1)
# FINAL TIMEOUT FALLBACK: After 40 seconds, proceed with whatever is loaded
if not ai_ready:
elapsed = int(time.time() - (overall_deadline - 40))
if elapsed >= 40:
self.logger.warning(f"⏱️ 40s timeout reached - proceeding with loaded content")
print(f" ⏱️ Timeout (40s) - scraping loaded content")
ai_ready = True # Proceed anyway
else:
self.logger.warning("AI completion not detected (proceeding anyway)")
print(f" ⚠️ No completion indicator (proceeding)")
# JavaScript Injection (DOM Marker & Extraction)
print(f" 📚 Injecting Markers & Extracting Sources...")
self.logger.debug("Starting JavaScript DOM injection...")
try:
# Inject citation selectors into JavaScript
script_with_selectors = DOM_INJECTION_SCRIPT.replace(
'%CITATION_SELECTORS%',
json.dumps(CITATION_SELECTORS)
)
data = self.page.evaluate(script_with_selectors)
self.logger.debug("JavaScript injection successful")
except Exception as e:
# Check for browser closed
self.logger.error(f"JavaScript injection failed: {e}")
if "browser has been closed" in str(e).lower() or "target closed" in str(e).lower():
return {
"success": False,
"error": "BROWSER_CLOSED_BY_USER",
"message": "Browser wurde vom User geschlossen"
}
return {"success": False, "error": f"JS Injection failed: {e}"}
if 'error' in data:
self.logger.error(f"JS script returned error: {data['error']}")
return {"success": False, "error": data['error']}
html_content = data['html']
citations = data['citations']
self.logger.debug(f"DOM injection: {len(citations)} citation groups")
# SIDEBAR FALLBACK: If DOM injection returned no citations
if len(citations) == 0:
self.logger.info("No citations from DOM, triggering sidebar fallback...")
print(f" 📌 No citation buttons, trying sidebar...")
fallback_sources = self._extract_sidebar_fallback()
if fallback_sources:
# Create single citation group with all sidebar sources
citations = [{
'marker_id': 0,
'sources': fallback_sources
}]
self.logger.info(f"✅ Sidebar fallback: {len(fallback_sources)} sources")
print(f" ✅ Sidebar fallback: {len(fallback_sources)} sources")
else:
self.logger.warning("No sources found (DOM + sidebar both empty)")
print(f" ⚠️ No sources found (DOM + sidebar both empty)")
# HTML Cleanup
self.logger.debug("Cleaning HTML content...")
html_cleaned = self._clean_html_pre_processing(html_content)
# Convert to Markdown
print(f" 🔄 Converting HTML to Markdown...")
self.logger.debug("Converting HTML to Markdown...")
if ConversionOptions:
options = ConversionOptions(
heading_style="atx",
list_indent_width=2,
bullets="*+- ",
wrap=False
)
markdown = convert(html_cleaned, options)
else:
markdown = convert(html_cleaned)
# Post-Processing (Text Cleanup)
self.logger.debug("Starting post-processing...")
# Entferne Highlighting-Marker (==), die Google/Converter erzeugt
markdown = markdown.replace('==', '')
# Entferne Base64 Bilder
markdown = re.sub(r'!\[[^\]]*\]\(data:image/[^)]+\)', '', markdown)
# Entferne leere Links
markdown = re.sub(r'\[\]\([^)]+\)', '', markdown)
# RADIKALER CUT-OFF: Alles ab dem AI-Disclaimer entfernen
cut_off_markers = [
'KI-Antworten können Fehler enthalten',
'AI-generated answers may contain mistakes',
'Öffentlicher Link wird erstellt'
]
for marker in cut_off_markers:
if marker in markdown:
markdown = markdown.split(marker)[0]
self.logger.debug(f"Cut off content at marker: {marker[:30]}...")
# SMART LINE MERGING (Fix broken sentences)
markdown = re.sub(r'([^\.\!\?\:\;\n])\n+\s*(\*\*)', r'\1 \2', markdown)
markdown = re.sub(r'([^\.\!\?\:\;\n])\n+\s*([a-zäöü])', r'\1 \2', markdown)
# Finales Trimmen
markdown = markdown.strip()
# Entferne alleinstehende Punkte auf eigener Zeile (nach dem Cut-off)
markdown = re.sub(r'^\s*\.\s*$', '', markdown, flags=re.MULTILINE)
# Leere Zeilen reduzieren
markdown = re.sub(r'\n{3,}', '\n\n', markdown).strip()
# Citations einfügen
print(f" 📌 Embedding {len(citations)} citations...")
self.logger.debug(f"Embedding {len(citations)} citation groups...")
markdown, sources = self._embed_citations(markdown, citations)
self.logger.debug(f"Total sources embedded: {len(sources)}")
# Quellenverzeichnis anhängen
if sources:
self.logger.debug("Appending sources section...")
markdown += "\n\n---\n\n## Sources:\n\n"
for i, source in enumerate(sources, 1):
markdown += f"[{i}] {source.get('title', 'Link')} \n{source.get('url')}\n\n"
self.logger.info(f"Scraping completed successfully - {len(sources)} sources, {len(markdown)} chars")
return {
"success": True,
"markdown": markdown,
"sources": sources,
"source_url": url,
"query": query
}
# =============================================================================
# CLI ENTRY POINT
# =============================================================================
def main():
parser = argparse.ArgumentParser(description="Google AI Mode Search")
# Input Arguments
parser.add_argument("--query", type=str, help="Full search query")
parser.add_argument("--city", type=str, help="City name (e.g. 'Münster')")
parser.add_argument("--plz", type=str, help="Postal code")
parser.add_argument("--topic", type=str, default="Mietspiegel 2026", help="Topic for constructed query")
# Options
parser.add_argument("--output", type=str, help="Custom output filename")
parser.add_argument("--show-browser", action="store_true", help="Run browser visibly (for debugging or captcha solving)")
parser.add_argument("--json", action="store_true", help="Save raw JSON alongside Markdown")
parser.add_argument("--debug", action="store_true", help="Enable verbose debug logging to logs/ folder")
parser.add_argument("--save", action="store_true", help="Save results to skill results/ folder instead of current directory")
args = parser.parse_args()
# Query Construction Logic
query = ""
if args.query:
query = args.query
elif args.city:
plz_part = f" {args.plz}" if args.plz else ""
query = f"{args.topic} {args.city}{plz_part}"
else:
print("❌ Error: You must provide either --query OR --city")
parser.print_help()
sys.exit(1)
print("=" * 60)
print(f"🚀 GOOGLE AI MODE SEARCH")
print(f" Query: '{query}'")
print(f" Mode: {'Visible' if args.show_browser else 'Headless'}")
if args.debug:
print(f" Debug: Enabled (logs will be saved)")
if args.save:
print(f" Save: Results folder")
print("=" * 60)
# Initialize logger
logger = get_logger(debug=args.debug)
logger.info(f"Starting search for: {query}")
logger.debug(f"Arguments: show_browser={args.show_browser}, debug={args.debug}, save={args.save}")
scraper = GoogleAIScraper(headless=not args.show_browser, logger=logger)
try:
scraper.start()
result = scraper.scrape(query)
if result['success']:
print("\n✅ SEARCH SUCCESSFUL")
print("-" * 60)
logger.info("Search completed successfully")
# Filename Generation
if args.output:
# User specified output path
out_path = Path(args.output)
logger.debug(f"Using custom output path: {out_path}")
elif args.save:
# Save to skill results/ folder with timestamp
RESULTS_DIR.mkdir(exist_ok=True)
timestamp = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
safe_name = re.sub(r'[^a-zA-Z0-9]', '_', query[:40]).strip('_')
out_path = RESULTS_DIR / f"{timestamp}_{safe_name}.md"
logger.debug(f"Saving to results folder: {out_path}")
else:
# Current directory (default)
safe_name = re.sub(r'[^a-zA-Z0-9]', '_', query[:40]).strip('_')
out_path = Path(f"result_{safe_name}.md")
logger.debug(f"Saving to current directory: {out_path}")
# Write Markdown
logger.debug(f"Writing markdown to: {out_path}")
with open(out_path, 'w', encoding='utf-8') as f:
f.write(result['markdown'])
print(f"📄 Saved Markdown: {out_path}")
logger.info(f"Markdown saved: {out_path}")
# Write JSON if requested
if args.json:
json_path = out_path.with_suffix('.json')
logger.debug(f"Writing JSON to: {json_path}")
with open(json_path, 'w', encoding='utf-8') as f:
json.dump(result, f, indent=2, ensure_ascii=False)
print(f"💾 Saved JSON: {json_path}")
logger.info(f"JSON saved: {json_path}")
# Preview (First 500 chars)
print("\n--- PREVIEW ---")
print(result['markdown'][:500] + "\n...")
else:
print("\n❌ SEARCH FAILED")
print(f"Error: {result.get('error')}")
print(f"Message: {result.get('message', '')}")
if result.get('suggestion'):
print(f"Suggestion: {result.get('suggestion')}")
logger.error(f"Search failed: {result.get('error')} - {result.get('message', '')}")
# Return specific exit codes for different errors
if result.get('error') == 'CAPTCHA_REQUIRED':
sys.exit(2) # Special exit code for captcha
elif result.get('error') == 'BROWSER_CLOSED_BY_USER':
sys.exit(3)
elif result.get('error') == 'AI_MODE_NOT_AVAILABLE':
sys.exit(4) # AI Mode not available in region
else:
sys.exit(1)
except KeyboardInterrupt:
print("\n⚠️ Aborted by User")
logger.warning("Search aborted by user (Ctrl+C)")
sys.exit(130)
except Exception as e:
print(f"\n❌ Unexpected Error: {e}")
logger.exception("Unexpected error occurred")
import traceback
traceback.print_exc()
sys.exit(1)
finally:
scraper.stop()
if logger.debug_enabled and logger.log_file:
print(f"\n📋 Debug log saved: {logger.log_file}")
if __name__ == "__main__":
main()
#!/usr/bin/env python3
"""
Environment Setup for Google AI Mode Skill
Manages virtual environment and dependencies automatically
"""
import os
import sys
import subprocess
import venv
from pathlib import Path
class SkillEnvironment:
"""Manages skill-specific virtual environment"""
def __init__(self):
# Skill directory paths
self.skill_dir = Path(__file__).parent.parent
self.venv_dir = self.skill_dir / ".venv"
self.requirements_file = self.skill_dir / "requirements.txt"
# Python executable in venv
if os.name == 'nt': # Windows
self.venv_python = self.venv_dir / "Scripts" / "python.exe"
self.venv_pip = self.venv_dir / "Scripts" / "pip.exe"
else: # Unix/Linux/Mac
self.venv_python = self.venv_dir / "bin" / "python"
self.venv_pip = self.venv_dir / "bin" / "pip"
def ensure_venv(self) -> bool:
"""Ensure virtual environment exists and is set up"""
# Check if we're already in the correct venv
if self.is_in_skill_venv():
print("✅ Already running in skill virtual environment")
return True
# Create venv if it doesn't exist
if not self.venv_dir.exists():
print(f"🔧 Creating virtual environment in {self.venv_dir.name}/")
try:
venv.create(self.venv_dir, with_pip=True)
print("✅ Virtual environment created")
except Exception as e:
print(f"❌ Failed to create venv: {e}")
return False
# Install/update dependencies
if self.requirements_file.exists():
print("📦 Installing dependencies...")
try:
# Upgrade pip first
subprocess.run(
[str(self.venv_pip), "install", "--upgrade", "pip"],
check=True,
capture_output=True,
text=True
)
# Install requirements
result = subprocess.run(
[str(self.venv_pip), "install", "-r", str(self.requirements_file)],
check=True,
capture_output=True,
text=True
)
print("✅ Dependencies installed")
# Install Chrome for Patchright (not Chromium!)
# Using real Chrome ensures cross-platform reliability and consistent browser fingerprinting
# See: https://github.com/Kaliiiiiiiiii-Vinyzu/patchright-python#anti-detection
print("🌐 Installing Google Chrome for Patchright...")
try:
subprocess.run(
[str(self.venv_python), "-m", "patchright", "install", "chrome"],
check=True,
capture_output=True,
text=True
)
print("✅ Chrome installed")
except subprocess.CalledProcessError as e:
print(f"⚠️ Warning: Failed to install Chrome: {e}")
print(" You may need to run manually: python -m patchright install chrome")
print(" Chrome is required (not Chromium) for reliability!")
return True
except subprocess.CalledProcessError as e:
print(f"❌ Failed to install dependencies: {e}")
print(f" Output: {e.output if hasattr(e, 'output') else 'No output'}")
return False
else:
print("⚠️ No requirements.txt found, skipping dependency installation")
return True
def is_in_skill_venv(self) -> bool:
"""Check if we're already running in the skill's venv"""
if hasattr(sys, 'real_prefix') or (hasattr(sys, 'base_prefix') and sys.base_prefix != sys.prefix):
# We're in a venv, check if it's ours
venv_path = Path(sys.prefix)
return venv_path == self.venv_dir
return False
def get_python_executable(self) -> str:
"""Get the correct Python executable to use"""
if self.venv_python.exists():
return str(self.venv_python)
return sys.executable
def run_script(self, script_name: str, args: list = None) -> int:
"""Run a script with the virtual environment"""
script_path = self.skill_dir / "scripts" / script_name
if not script_path.exists():
print(f"❌ Script not found: {script_path}")
return 1
# Ensure venv is set up
if not self.ensure_venv():
print("❌ Failed to set up environment")
return 1
# Build command
cmd = [str(self.venv_python), str(script_path)]
if args:
cmd.extend(args)
print(f"🚀 Running: {script_name} with venv Python")
try:
# Run the script with venv Python
result = subprocess.run(cmd)
return result.returncode
except Exception as e:
print(f"❌ Failed to run script: {e}")
return 1
def activate_instructions(self) -> str:
"""Get instructions for manual activation"""
if os.name == 'nt':
# Windows supports both CMD and PowerShell
activate_cmd = self.venv_dir / "Scripts" / "activate.bat"
activate_ps = self.venv_dir / "Scripts" / "Activate.ps1"
return f"CMD: {activate_cmd}\nPowerShell: {activate_ps}"
else:
activate = self.venv_dir / "bin" / "activate"
return f"source {activate}"
def main():
"""Main entry point for environment setup"""
import argparse
parser = argparse.ArgumentParser(
description='Setup Google AI Mode skill environment'
)
parser.add_argument(
'--check',
action='store_true',
help='Check if environment is set up'
)
parser.add_argument(
'--run',
help='Run a script with the venv (e.g., --run ask_question.py)'
)
parser.add_argument(
'args',
nargs='*',
help='Arguments to pass to the script'
)
args = parser.parse_args()
env = SkillEnvironment()
if args.check:
if env.venv_dir.exists():
print(f"✅ Virtual environment exists: {env.venv_dir}")
print(f" Python: {env.get_python_executable()}")
print(f" To activate manually: {env.activate_instructions()}")
else:
print(f"❌ No virtual environment found")
print(f" Run setup_environment.py to create it")
return
if args.run:
# Run a script with venv
return env.run_script(args.run, args.args)
# Default: ensure environment is set up
if env.ensure_venv():
print("\n✅ Environment ready!")
print(f" Virtual env: {env.venv_dir}")
print(f" Python: {env.get_python_executable()}")
print(f"\nTo activate manually: {env.activate_instructions()}")
print(f"Or run scripts directly: python setup_environment.py --run script_name.py")
else:
print("\n❌ Environment setup failed")
return 1
if __name__ == "__main__":
sys.exit(main() or 0)Related skills
FAQ
What does google-ai-mode-skill do?
google-ai-mode-skill is a Claude Code skill for ai & agent building.
When should I use google-ai-mode-skill?
When you need to helps with ai & agent building tasks during AI-assisted development., or when google-ai-mode-skill is a claude code skill for ai & agent building.
What are the main capabilities?
google-ai-mode-skill; AI & Agent Building; AI-coding skill.