
Blog Google
- 1.6k installs
- 1.6k repo stars
- Updated July 23, 2026
- agricidaniel/claude-blog
Tune blog posts for Google Search performance—metadata, structure, indexing hygiene, and SERP-oriented copy—when organic discovery is a primary launch channel.
About
blog-google in agricidaniel/claude-blog applies Google Search optimization practices to blog drafts, improving titles, meta descriptions, heading structure, and on-page signals. It helps publishers increase eligible impressions, clicks, and sustained organic traffic from traditional search results.
- Google Search optimization
- Metadata and headings
- Indexing best practices
- SERP-oriented drafting
- Organic launch channel
Blog Google by the numbers
- 1,643 all-time installs (skills.sh)
- +41 installs in the week ending Aug 5, 2026 (Skillselion tracking)
- Ranked #359 of 1,879 Marketing & SEO skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/agricidaniel/claude-blog --skill blog-googleAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 1.6k |
|---|---|
| repo stars | ★ 1.6k |
| Last updated | July 23, 2026 |
| Repository | agricidaniel/claude-blog ↗ |
What it does
Tune blog posts for Google Search performance—metadata, structure, indexing hygiene, and SERP-oriented copy—when organic discovery is a primary launch channel.
Files
Blog Google: Google API Data for Blog Performance
Direct access to Google's SEO APIs for blog performance analysis. Provides real Chrome user metrics, indexation status, search performance, entity analysis, YouTube video discovery, keyword volumes, and PDF/HTML performance reports.
All APIs are free at normal usage levels. Setup requires a Google Cloud project with an API key and/or service account.
Prerequisites
Always check credentials before running any command:
python3 skills/blog-google/scripts/run.py google_auth --check --jsonConfig file: ~/.config/claude-seo/google-api.json (shared with claude-seo)
{
"api_key": "AIzaSy...",
"oauth_client_path": "/path/to/client_secret.json",
"default_property": "sc-domain:example.com",
"ga4_property_id": "properties/123456789",
"ads_developer_token": "...",
"ads_customer_id": "123-456-7890"
}If missing, read references/auth-setup.md and walk the user through setup.
Credential Tiers
| Tier | Detection | Available Commands |
|---|---|---|
| 0 (API Key) | api_key present | pagespeed, crux, crux-history, youtube, nlp |
| 1 (OAuth/SA) | + OAuth token or service account | Tier 0 + gsc, inspect, index |
| 2 (Full) | + ga4_property_id configured | Tier 1 + ga4 |
| 3 (Ads) | + ads_developer_token + ads_customer_id | Tier 2 + keywords |
Always communicate the detected tier before running commands.
Quick Reference
| Command | What it does | Tier |
|---|---|---|
/blog google setup | Check/configure API credentials | : |
/blog google pagespeed <url> | PSI Lighthouse + CrUX field data | 0 |
/blog google crux <url> | CrUX field data only (p75 metrics) | 0 |
/blog google crux-history <url> | 25-week CWV trend analysis | 0 |
/blog google youtube <query> | YouTube video search (views, likes, duration) | 0 |
/blog google nlp <url-or-text> | NLP entity extraction + sentiment | 0 |
/blog google gsc <property> | Search Console: clicks, impressions, CTR, position | 1 |
/blog google inspect <url> | URL Inspection: index status, canonical | 1 |
/blog google index <url> | Submit URL to Indexing API | 1 |
/blog google ga4 [property-id] | GA4 organic traffic report | 2 |
/blog google keywords <seed> | Keyword ideas from Google Ads Keyword Planner | 3 |
/blog google report <type> | PDF/HTML performance report | : |
/blog google quotas | Show rate limits for all APIs | : |
---
PageSpeed + CrUX
/blog google pagespeed <url>
Combined Lighthouse lab data + CrUX field data for a published blog post.
Script: python3 skills/blog-google/scripts/run.py pagespeed_check <url> --json Reference: references/api-reference.md
Output merges lab scores (point-in-time Lighthouse) with field data (28-day Chrome user metrics). CrUX tries URL-level first, falls back to origin-level.
/blog google crux <url>
CrUX field data only (no Lighthouse run). Faster.
Script: python3 skills/blog-google/scripts/run.py pagespeed_check <url> --crux-only --json
/blog google crux-history <url>
25-week CrUX History trends. Shows whether CWV metrics are improving, stable, or degrading.
Script: python3 skills/blog-google/scripts/run.py crux_history <url> --json
---
Search Console
/blog google gsc <property>
Search Analytics: clicks, impressions, CTR, position for last 28 days.
Script: python3 skills/blog-google/scripts/run.py gsc_query --property <property> --json Default: 28 days, dimensions=query,page, type=web, limit=1000.
Includes quick-win detection: queries at position 4-10 with high impressions.
/blog google inspect <url>
URL Inspection: real indexation status from Google.
Script: python3 skills/blog-google/scripts/run.py gsc_inspect <url> --json
Returns: verdict (PASS/FAIL), coverage state, robots.txt status, indexing state, page fetch state, canonical selection, mobile usability, rich results.
For batch inspection: python3 skills/blog-google/scripts/run.py gsc_inspect --batch <file> --json
---
Indexing API
/blog google index <url>
Notify Google of a URL update. Submit new blog posts for faster indexation.
Script: python3 skills/blog-google/scripts/run.py indexing_notify <url> --json Reference: references/api-reference.md
The Indexing API is officially for JobPosting and BroadcastEvent/VideoObject pages. Always inform the user of this restriction. Daily quota: 200 publish requests.
For batch: python3 skills/blog-google/scripts/run.py indexing_notify --batch <file> --json
---
GA4 Traffic
/blog google ga4 [property-id]
Organic traffic report: daily sessions, users, pageviews, bounce rate, engagement.
Script: python3 skills/blog-google/scripts/run.py ga4_report --property <id> --json Default: 28 days, filtered to Organic Search channel group.
For top landing pages: python3 skills/blog-google/scripts/run.py ga4_report --property <id> --report top-pages --json
---
YouTube (Video Discovery)
YouTube mentions have the strongest AI visibility correlation (0.737, Ahrefs 75K brands). Free, API key only. Used by blog-write and blog-rewrite for video embedding.
/blog google youtube <query>
Search YouTube for videos relevant to a blog topic.
Script: python3 skills/blog-google/scripts/run.py youtube_search search "<query>" --json Quota: 100 units per search (10,000 units/day free).
Returns: title, channel, views, likes, duration, description, tags.
For video details + comments: python3 skills/blog-google/scripts/run.py youtube_search video <video_id> --json
---
NLP Content Analysis
Google's own entity/sentiment analysis. Enhances E-E-A-T scoring for blog content.
/blog google nlp <url-or-text>
Full NLP analysis: entities, sentiment, content classification.
Script: python3 skills/blog-google/scripts/run.py nlp_analyze --url <url> --json Free tier: 5,000 units/month. Requires billing enabled on GCP project.
For entity extraction only: python3 skills/blog-google/scripts/run.py nlp_analyze --url <url> --features entities --json
---
Keyword Research (Google Ads)
Gold-standard keyword volume data. Requires Google Ads account (Tier 3).
/blog google keywords <seed>
Generate keyword ideas from seed terms for blog topic research.
Script: python3 skills/blog-google/scripts/run.py keyword_planner ideas "<seed>" --json
For volume lookup: python3 skills/blog-google/scripts/run.py keyword_planner volume "<kw1>,<kw2>" --json
---
Reports
/blog google report <type>
Generate a professional PDF/HTML report with charts.
Script: python3 skills/blog-google/scripts/run.py google_report --type <type> --data <json> --domain <domain> --format pdf
| Type | Input | Output |
|---|---|---|
cwv-audit | PSI + CrUX + CrUX History data | Core Web Vitals audit with gauges, timelines |
gsc-performance | GSC query data | Search Console report with query tables |
indexation | Batch inspection data | Indexation status with coverage donut |
full | All data combined | Comprehensive Google SEO report |
Note: PDF generation requires system libraries: sudo apt install libpango1.0-dev libcairo2-dev. Falls back to HTML if weasyprint is unavailable.
---
Rate Limits
| API | Per-Minute | Per-Day | Auth |
|---|---|---|---|
| PSI v5 | 240 QPM | 25,000 QPD | API Key |
| CrUX + History | 150 QPM (shared) | Unlimited | API Key |
| GSC Search Analytics | 1,200 QPM/site | 30M QPD | Service Account |
| GSC URL Inspection | 600 QPM | 2,000 QPD/site | Service Account |
| Indexing API | 380 RPM | 200 publish/day | Service Account |
| GA4 Data API | 10 concurrent (50 for 360) | 200K Core Tokens/day (2M for 360) | Service Account |
| YouTube Data | : | 10,000 units/day | API Key |
| NLP API | : | 5,000 units/month | API Key (billing) |
Read references/rate-limits-quotas.md for detailed quota management.
Blog Workflow Integration
This skill is both user-invocable (/blog google pagespeed) and callable internally by other blog sub-skills:
- blog-seo-check: Runs PSI + CrUX on published post URL for live CWV data
- blog-rewrite: NLP entity analysis to identify E-E-A-T entity gaps
- blog-geo: GSC performance data for real search appearance insights
- blog-audit: Batch CWV + indexation checks across all published blog URLs
- blog-write / blog-rewrite: YouTube search for video embedding
Falls back gracefully when credentials are not configured.
Technical Notes
- INP replaced FID on March 12, 2024. Never reference FID.
- CLS values from CrUX are string-encoded (e.g., "0.05"). Scripts handle parsing.
- CrUX 404 = insufficient Chrome traffic, not an auth error.
- Search Analytics data has 2-3 day lag.
- Indexing API is officially for JobPosting/BroadcastEvent pages only.
- All Google APIs used are FREE at normal usage levels.
Error Handling
| Scenario | Action |
|---|---|
| No credentials configured | Run /blog google setup. List Tier 0 commands (API key only). |
| Service account lacks GSC access | Add client_email to GSC > Settings > Users > Add. |
| CrUX data unavailable (404) | Insufficient Chrome traffic. Use PSI lab data as fallback. |
| GA4 property not found | Find property ID in GA4 Admin > Property Details. |
| Indexing API quota exceeded | 200/day limit. Prioritize most important URLs. |
| Rate limit (429) | Wait and retry with exponential backoff. |
Core Web Vitals Audit
URL/Origin: {target} Strategy: {strategy}
CrUX Field Data (28-day rolling average)
Real Chrome user experience data from the Chrome UX Report.
| Metric | p75 Value | Rating | Good Threshold | Distribution |
|---|---|---|---|---|
| LCP | {lcp_value} | {lcp_rating} | ≤ 2,500ms | Good: {lcp_good}% / NI: {lcp_ni}% / Poor: {lcp_poor}% |
| INP | {inp_value} | {inp_rating} | ≤ 200ms | Good: {inp_good}% / NI: {inp_ni}% / Poor: {inp_poor}% |
| CLS | {cls_value} | {cls_rating} | ≤ 0.1 | Good: {cls_good}% / NI: {cls_ni}% / Poor: {cls_poor}% |
| FCP | {fcp_value} | {fcp_rating} | ≤ 1,800ms | Good: {fcp_good}% / NI: {fcp_ni}% / Poor: {fcp_poor}% |
| TTFB | {ttfb_value} | {ttfb_rating} | ≤ 800ms | Good: {ttfb_good}% / NI: {ttfb_ni}% / Poor: {ttfb_poor}% |
Collection Period: {collection_start} to {collection_end}
Lighthouse Lab Scores
| Category | Score |
|---|---|
| Performance | {perf_score}/100 |
| Accessibility | {a11y_score}/100 |
| Best Practices | {bp_score}/100 |
| SEO | {seo_score}/100 |
CrUX History Trends (25-week)
| Metric | Direction | Change | Earliest → Latest |
|---|
{trends_table}
Top Opportunities
| Opportunity | Estimated Savings |
|---|
{opportunities_table}
Recommendations
{recommendations}
--- CrUX data updated daily ~04:00 UTC. 28-day rolling average. INP replaced FID as the responsiveness Core Web Vital on March 12, 2024. Generated {timestamp}.
Google Search Console Performance Report
Property: {property} Date Range: {start_date} - {end_date} Search Type: {search_type}
Summary
| Metric | Value |
|---|---|
| Total Clicks | {total_clicks} |
| Total Impressions | {total_impressions} |
| Average CTR | {avg_ctr}% |
| Average Position | {avg_position} |
Top Queries
| # | Query | Clicks | Impressions | CTR | Position |
|---|
{queries_table}
Top Pages
| # | Page | Clicks | Impressions | CTR | Position |
|---|
{pages_table}
Quick Wins (Position 4-10, High Impressions)
These queries rank on page 1 but below position 3. A small ranking improvement could yield significant traffic gains.
| Query | Position | Impressions | Clicks | CTR | Opportunity |
|---|
{quick_wins_table}
Device Breakdown
| Device | Clicks | Impressions | CTR | Position |
|---|
{device_table}
--- Data freshness: Search Analytics has a 2-3 day lag. Data available for ~16 months. Generated {timestamp} via Google Search Console API.
URL Indexation Status Report
Property: {property} URLs Inspected: {total_urls}
Summary
| Status | Count | Percentage |
|---|---|---|
| Indexed (PASS) | {pass_count} | {pass_pct}% |
| Not Indexed (FAIL) | {fail_count} | {fail_pct}% |
| Neutral | {neutral_count} | {neutral_pct}% |
| Errors | {error_count} | {error_pct}% |
Detailed Results
| URL | Verdict | Coverage State | Fetch State | Google Canonical | Last Crawl |
|---|
{results_table}
Canonical Mismatches
URLs where Google selected a different canonical than declared:
| URL | User Canonical | Google Canonical |
|---|
{canonical_mismatches_table}
Common Issues
| Issue | Count | Priority | Action |
|---|
{issues_table}
Rich Results Detected
| URL | Rich Result Type | Status |
|---|
{rich_results_table}
--- URL Inspection API: 2,000 inspections/day per site, 600/min. Generated {timestamp} via Google Search Console URL Inspection API.
Blog Google - API Reference
Consolidated reference for all Google APIs used by the blog-google skill.
---
PageSpeed Insights v5
Endpoint: GET https://pagespeedonline.googleapis.com/pagespeedonline/v5/runPagespeed (canonical) Legacy hostname also works: https://www.googleapis.com/pagespeedonline/v5/runPagespeed
| Param | Type | Description |
|---|---|---|
url | string | Required. URL to analyze |
category | string | ACCESSIBILITY, BEST_PRACTICES, PERFORMANCE, SEO (can specify multiple) |
strategy | string | DESKTOP or MOBILE (default) |
key | string | API key (optional but recommended) |
Response contains loadingExperience (URL-level CrUX), originLoadingExperience (origin CrUX), and lighthouseResult with category scores and audit details.
Note: Google has signaled intent to remove CrUX from PSI, but as of April 2026 CrUX field data (loadingExperience, originLoadingExperience) is still returned in PSI responses. The standalone CrUX API is the recommended long-term solution for field data; use PSI primarily for Lighthouse lab scores.
---
CrUX API (Daily)
Endpoint: POST https://chromeuxreport.googleapis.com/v1/records:queryRecord?key={API_KEY}
{
"origin": "https://example.com",
"formFactor": "PHONE",
"metrics": ["largest_contentful_paint", "interaction_to_next_paint", "cumulative_layout_shift"]
}originandurlare mutually exclusive. Useoriginfor site-wide,urlfor a specific page.formFactor:DESKTOP,PHONE,TABLET(omit for all).- Each metric returns
histogram(density buckets),percentiles.p75, andcategory. - CLS p75 is a string (e.g.,
"0.05"not0.05). Always parse as float. - 404 = insufficient Chrome traffic (not an auth error).
- Updated daily on a best-effort basis with ~2-day lag. No guaranteed update time; timezone is PST.
---
CrUX History API (Weekly)
Endpoint: POST https://chromeuxreport.googleapis.com/v1/records:queryHistoryRecord?key={API_KEY}
Same request format as CrUX API. Returns up to 40 weekly collection periods (~10 months) as timeseries arrays (p75s[], densities[]). Default is 25; configurable via collectionPeriodCount parameter (range: 1-40).
- Updated Mondays ~04:00 UTC.
- Each period = 28-day rolling average ending on a Sunday.
- Watch for
"NaN"strings in densities andnullin percentiles for ineligible periods.
---
Core Web Vitals Thresholds
Current as of March 2026. INP replaced FID on March 12, 2024.
Core Web Vitals (the 3 official CWV):
| Metric | Good | Needs Improvement | Poor |
|---|---|---|---|
| LCP | ≤ 2,500ms | 2,500-4,000ms | > 4,000ms |
| INP | ≤ 200ms | 200-500ms | > 500ms |
| CLS | ≤ 0.1 | 0.1-0.25 | > 0.25 |
Diagnostic metrics (not CWV - informational only):
| Metric | Good | Needs Improvement | Poor |
|---|---|---|---|
| FCP | ≤ 1,800ms | 1,800-3,000ms | > 3,000ms |
| TTFB | ≤ 800ms | 800-1,800ms | > 1,800ms |
---
GSC Search Analytics
Endpoint: POST https://www.googleapis.com/webmasters/v3/sites/{siteUrl}/searchAnalytics/query
Request Body
| Field | Type | Description |
|---|---|---|
startDate | string | Required. YYYY-MM-DD |
endDate | string | Required. YYYY-MM-DD |
dimensions | string[] | query, page, country, device, date, searchAppearance |
type | string | web, image, video, news, discover, googleNews |
dimensionFilterGroups | object[] | Filter groups with dimension, operator, expression |
rowLimit | int | 1-25000 (default: 1000) |
startRow | int | Pagination offset (default: 0) |
dataState | string | final (default), all, hourly_all (April 2025, requires HOUR dimension) |
Filter Operators
contains, equals, notContains, notEquals, includingRegex, excludingRegex
Response Fields
Each row: keys[], clicks, impressions, ctr, position.
- Data lag by
dataState:final= ~2-3 days;all= shorter lag;hourly_all= few hours (April 2025). Retention: ~16 months. - Country codes are ISO 3166-1 alpha-3 (e.g.,
USA,GBR).
---
GSC URL Inspection
Endpoint: POST https://searchconsole.googleapis.com/v1/urlInspection/index:inspect
{
"inspectionUrl": "https://example.com/blog/post",
"siteUrl": "sc-domain:example.com",
"languageCode": "en"
}Key Response Fields (indexStatusResult)
| Field | Values |
|---|---|
verdict | PASS, FAIL, NEUTRAL, PARTIAL |
coverageState | Human-readable coverage description |
robotsTxtState | ALLOWED, DISALLOWED |
indexingState | INDEXING_ALLOWED, BLOCKED_BY_META_TAG, BLOCKED_BY_HTTP_HEADER |
pageFetchState | SUCCESSFUL, SOFT_404, BLOCKED_ROBOTS_TXT, NOT_FOUND, SERVER_ERROR |
lastCrawlTime | ISO 8601 timestamp |
googleCanonical | URL Google selected as canonical |
crawledAs | DESKTOP, MOBILE |
---
GA4 Data API v1beta
Base URL: https://analyticsdata.googleapis.com/v1beta
runReport Overview
Key fields: property, dimensions[], metrics[], dateRanges[], dimensionFilter, orderBys[], limit.
Blog-Relevant Dimensions
date, landingPage, pagePath, pageTitle, sessionDefaultChannelGroup, sessionSource, sessionMedium, country, deviceCategory
Blog-Relevant Metrics
sessions, totalUsers, screenPageViews, bounceRate, averageSessionDuration, engagementRate, keyEvents
Organic Traffic Filter
{
"filter": {
"fieldName": "sessionDefaultChannelGroup",
"stringFilter": { "matchType": "EXACT", "value": "Organic Search" }
}
}Uses token-based quotas (200,000 Core Tokens/day per standard property; 2M for 360). Set returnPropertyQuota: true to monitor consumption.
---
Cloud Natural Language API
Endpoint: POST https://language.googleapis.com/v2/documents:annotateText?key={API_KEY}
| Feature | What It Does | Blog Use |
|---|---|---|
extractEntities | People, orgs, places with salience scores | Topic coverage depth, entity optimization |
extractDocumentSentiment | Document + sentence-level sentiment | Content tone assessment |
classifyText | Map content to 700+ Google categories | Topic relevance verification |
moderateText | Detect harmful/sensitive content categories | Content safety screening |
Each entity includes name, type, salience (0-1), sentiment, and metadata (Wikipedia URL, Knowledge Graph MID).
Pricing: 5,000 free units/month for entities and sentiment. Requires billing enabled.
---
YouTube Data API v3
YouTube mentions correlate strongly with AI citation visibility (GEO research).
| Method | Quota Cost | Description |
|---|---|---|
search.list | 100 units | Search videos matching a query |
videos.list | 1 unit | Video details, statistics, tags |
channels.list | 1 unit | Channel info, subscriber count |
Default quota: 10,000 units/day (free). API key only, no OAuth needed.
---
Keyword Planner (Google Ads API)
Gold-standard source for keyword search volume. Methods: GenerateKeywordIdeas (suggestions from seeds), GenerateKeywordHistoricalMetrics (volume for specific keywords), and GenerateKeywordForecastMetrics (future projections). Returns volume, competition, CPC bids.
Current API version: v23.1 (released Feb 25, 2026). Monthly release cadence since Jan 2026. Any version below v20 is sunset. Update versioned endpoint paths accordingly.
- Without active ad spend, volumes are bucketed ranges ("1K-10K") not exact numbers
competitionmeasures advertiser competition, not organic difficulty- Requires: Google Ads Manager Account + Developer Token + OAuth credentials
Blog Google - Authentication Setup
Overview
Four credential tiers serve different API combinations:
| Tier | Credentials | APIs Unlocked |
|---|---|---|
| 0 | API Key only | PageSpeed Insights, CrUX, CrUX History, Knowledge Graph, YouTube Data |
| 1 | + Service Account | + Search Console, Indexing API |
| 2 | + GA4 property ID | + GA4 Data API |
| 3 | + Google Ads tokens | + Keyword Planner |
Step 1: Create a Google Cloud Project
1. Go to console.cloud.google.com 2. Click Select a project > New Project 3. Name it (e.g., "Claude Blog") and note the project ID 4. Select the project after creation
Step 2: Enable APIs
Navigate to APIs & Services > Library and enable:
| API | Required For |
|---|---|
| PageSpeed Insights API | Lighthouse lab data, CWV scores |
| Chrome UX Report API | CrUX field data + History |
| Google Search Console API | Search Analytics, URL Inspection, Sitemaps |
| Web Search Indexing API | Indexing API v3 (new post notifications) |
| Google Analytics Data API | GA4 organic traffic analysis |
| YouTube Data API v3 | Video research for GEO/AEO optimization |
| Cloud Natural Language API | Entity salience, sentiment, classification |
| Knowledge Graph Search API | Entity/brand verification |
Step 3: Create an API Key (Tier 0)
1. APIs & Services > Credentials > Create Credentials > API key 2. Click Restrict key > under API restrictions, select the APIs above 3. Copy the key (starts with AIzaSy...)
Step 4: Create a Service Account (Tier 1)
1. IAM & Admin > Service Accounts > Create Service Account 2. Name: claude-blog (or similar) 3. Skip optional permissions steps 4. Click on the created service account > Keys > Add Key > Create new key > JSON 5. Download and store securely (e.g., ~/.config/claude-seo/service_account.json)
Grant Search Console Access
1. Go to Google Search Console 2. Select your property > Settings > Users and permissions > Add user 3. Paste the service account client_email from the JSON file 4. Set permission: Full (read-only) or Owner (if using Indexing API)
Step 5: Set Up OAuth for Interactive Flows
1. APIs & Services > Credentials > Create Credentials > OAuth client ID 2. Application type: Desktop app 3. Download the client_secret_*.json file 4. Store at ~/.config/claude-seo/oauth_client.json
OAuth is needed for Keyword Planner and any user-consent flows.
Step 6: GA4 Property ID (Tier 2)
1. Go to Google Analytics 2. Admin > Property Access Management > Add users (+ icon) 3. Paste the service account client_email, set role: Viewer 4. Note the numeric property ID from Admin > Property Details (e.g., 123456789)
Step 7: Google Ads Credentials (Tier 3)
1. Create a Google Ads Manager Account at ads.google.com 2. Apply for a Developer Token at Google Ads API Center 3. Note: Without active ad spend, Keyword Planner returns bucketed ranges ("1K-10K")
Config File
Config is shared with claude-seo at ~/.config/claude-seo/google-api.json:
{
"service_account_path": "~/.config/claude-seo/service_account.json",
"api_key": "AIzaSy...",
"default_property": "sc-domain:example.com",
"ga4_property_id": "properties/123456789",
"ads_developer_token": "YOUR_DEV_TOKEN",
"ads_customer_id": "123-456-7890"
}GSC Property URL Formats
| Format | Example | When to Use |
|---|---|---|
| Domain property | sc-domain:example.com | All URLs on the domain (recommended) |
| URL-prefix property | https://example.com/ | Only that specific prefix |
Environment Variable Fallbacks
| Variable | Purpose |
|---|---|
GOOGLE_API_KEY | API key for PSI/CrUX/YouTube/NLP/Knowledge Graph |
GOOGLE_APPLICATION_CREDENTIALS | Path to service account JSON |
GA4_PROPERTY_ID | GA4 property (e.g., properties/123456789) |
GSC_PROPERTY | Default GSC property (e.g., sc-domain:example.com) |
Verify Setup
python scripts/google_auth.py --checkQuick Troubleshooting
| Error | Fix |
|---|---|
403 Forbidden on GSC | Service account email not added to property, or wrong permission level |
403 Forbidden on GA4 | Service account not added as Viewer in GA4 property |
404 Not Found on GSC | Wrong property URL format: use sc-domain: or include trailing slash |
404 Not Found on CrUX | Site has insufficient Chrome traffic (not a credentials issue) |
429 Rate Limit | Wait and retry with backoff. See rate-limits-quotas.md |
API not enabled | Enable the specific API in GCP Console > APIs & Services > Library |
Billing required | NLP API requires billing enabled (free tier still applies) |
Blog Google - Rate Limits & Quotas
Consolidated Quota Table
| API | Per-Minute | Per-Day | Cost | Auth Type | Scope |
|---|---|---|---|---|---|
| GSC Search Analytics | 1,200 QPM/user, 1,200 QPM/site | 30M QPD/project | Free | Service Account | Per user + per site |
| GSC URL Inspection | 600 QPM/site | 2,000 QPD/site | Free | Service Account | Per site |
| GSC Sitemaps | Standard | Standard | Free | Service Account | Per site |
| PageSpeed Insights v5 | 240 QPM | 25,000 QPD | Free | API Key | Per project |
| CrUX API | 150 QPM (shared) | Unlimited | Free | API Key | Per project |
| CrUX History API | 150 QPM (shared with CrUX) | Unlimited | Free | API Key | Per project |
| Indexing API | 380 RPM total, 180 read/min | 200 publish/day | Free | Service Account | Per project |
| GA4 Data API | 10 concurrent (50 for 360) | 200,000 Core Tokens/day (std); 2M (360) | Free | Service Account | Per property/project |
| Knowledge Graph | : | 100,000 QPD | Free | API Key | Per project |
| Custom Search | : | 10,000 QPD max | 100 free, $5/1K | API Key | Per project |
| Web Risk | 6,000 QPM | 100K/month | Free tier | API Key | Per project |
Key distinction: "Per site" quotas are scoped to a specific GSC property. "Per project" quotas are shared across all properties in a GCP project. "Per user" quotas are per authenticated user (service account).
Exponential Backoff Strategy
When receiving 429 or 5xx errors:
Attempt 1: wait 1 second
Attempt 2: wait 2 seconds
Attempt 3: wait 4 seconds
Attempt 4: wait 8 seconds
Attempt 5: wait 16 seconds
Max: give up after 5 retriesAdd random jitter (0-500ms) to each wait to avoid thundering herd.
Common Error Codes
| Code | Meaning | Applies To | Action |
|---|---|---|---|
| 400 | Bad request | All | Check URL format, request body |
| 401 | Unauthorized | Service Account APIs | Refresh credentials |
| 403 | Forbidden | GSC, GA4, Indexing | Check permissions (service account access) |
| 404 | Not found | CrUX, GSC | Insufficient traffic (CrUX) or invalid property (GSC) |
| 429 | Rate limited | All | Backoff and retry. Check Retry-After header. |
| 500 | Server error | All | Retry with backoff |
| 503 | Service unavailable | All | Retry with backoff |
Retry-After Header
Some Google APIs return a Retry-After header with 429 responses. When present, use this value (in seconds) instead of exponential backoff.
GA4 Token Budgeting
GA4 uses a token system rather than simple request counts:
- Simple 1-dimension, 1-metric report: ~1-5 tokens
- Complex multi-dimension, multi-metric: ~10-100 tokens
- Set
returnPropertyQuota: trueto monitor consumption - Daily limit: 200,000 Core Tokens/day per standard property (2,000,000 for 360)
- Hourly limit: 40,000 tokens/hour per property; 14,000/hour per project per property
- Concurrent: max 10 simultaneous requests (50 for 360)
- Source: developers.google.com/analytics/devguides/reporting/data/v1/quotas (updated 2026-03-26)
CrUX Shared Quota
The CrUX API and CrUX History API share the same 150 QPM quota per project. Plan accordingly if querying both APIs in the same workflow.
Cost Summary
All APIs used by blog-google are free at normal usage levels. No billing is required for:
- PSI, CrUX, CrUX History (API key, unlimited free)
- GSC (service account, 30M QPD)
- Indexing API (service account, 200 publish/day)
- GA4 Data API (service account, 200K tokens/day standard)
- Knowledge Graph (API key, 100K QPD)
Only Custom Search and Web Risk have paid tiers at high volumes.
#!/usr/bin/env python3
"""
CrUX History API for Core Web Vitals trends over time.
Fetches up to 25 weekly data points from the Chrome UX Report History API
and identifies improving, stable, or degrading trends per metric.
Usage:
python crux_history.py https://example.com
python crux_history.py https://example.com --form-factor PHONE --json
python crux_history.py https://example.com --origin
"""
import argparse
import json
import sys
from typing import Optional
from urllib.parse import urlparse
try:
import requests
except ImportError:
print("Error: requests library required. Install with: pip install requests")
sys.exit(1)
try:
from google_auth import get_api_key
except ImportError:
import os
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from google_auth import get_api_key
CRUX_HISTORY_ENDPOINT = "https://chromeuxreport.googleapis.com/v1/records:queryHistoryRecord"
CWV_THRESHOLDS = {
"largest_contentful_paint": {"good": 2500, "poor": 4000, "label": "LCP", "unit": "ms"},
"interaction_to_next_paint": {"good": 200, "poor": 500, "label": "INP", "unit": "ms"},
"cumulative_layout_shift": {"good": 0.1, "poor": 0.25, "label": "CLS", "unit": ""},
"first_contentful_paint": {"good": 1800, "poor": 3000, "label": "FCP", "unit": "ms"},
"experimental_time_to_first_byte": {"good": 800, "poor": 1800, "label": "TTFB", "unit": "ms"},
}
def query_history(
url_or_origin: str,
api_key: str,
form_factor: Optional[str] = None,
) -> dict:
"""
Query CrUX History API for weekly CWV trends.
Args:
url_or_origin: Full URL or origin.
api_key: Google API key.
form_factor: DESKTOP, PHONE, or TABLET. None for all.
Returns:
Dictionary with metrics timeseries, collection periods, and trend analysis.
"""
result = {
"target": url_or_origin,
"form_factor": form_factor or "ALL",
"metrics": {},
"collection_periods": [],
"trends": {},
"error": None,
}
parsed = urlparse(url_or_origin)
is_origin = parsed.path in ("", "/") and not parsed.query
body = {}
if is_origin:
body["origin"] = f"{parsed.scheme}://{parsed.netloc}"
else:
body["url"] = url_or_origin
if form_factor:
body["formFactor"] = form_factor.upper()
try:
resp = requests.post(
f"{CRUX_HISTORY_ENDPOINT}?key={api_key}",
json=body,
timeout=30,
)
if resp.status_code == 404:
target_type = "origin" if is_origin else "URL"
result["error"] = (
f"No CrUX history data for this {target_type}. "
"Insufficient Chrome traffic volume for eligibility."
)
return result
if resp.status_code == 429:
result["error"] = "CrUX API rate limit exceeded (150 QPM shared). Wait and retry."
return result
resp.raise_for_status()
data = resp.json()
except requests.exceptions.RequestException as e:
result["error"] = f"CrUX History API request failed: {e}"
return result
record = data.get("record", {})
# Collection periods
periods = record.get("collectionPeriods", [])
for period in periods:
first = period.get("firstDate", {})
last = period.get("lastDate", {})
result["collection_periods"].append({
"first": f"{first.get('year')}-{first.get('month', 0):02d}-{first.get('day', 0):02d}",
"last": f"{last.get('year')}-{last.get('month', 0):02d}-{last.get('day', 0):02d}",
})
# Metrics timeseries
for metric_name, metric_data in record.get("metrics", {}).items():
if metric_name not in CWV_THRESHOLDS:
continue
thresholds = CWV_THRESHOLDS[metric_name]
p75s_data = metric_data.get("percentilesTimeseries", {})
p75s_raw = p75s_data.get("p75s", [])
# Parse p75 values (CLS is string-encoded)
p75s = []
for val in p75s_raw:
if val is None:
p75s.append(None)
elif metric_name == "cumulative_layout_shift":
try:
p75s.append(float(str(val)))
except (ValueError, TypeError):
p75s.append(None)
else:
try:
p75s.append(int(val))
except (ValueError, TypeError):
try:
p75s.append(float(val))
except (ValueError, TypeError):
p75s.append(None)
# Distributions timeseries
histogram_ts = metric_data.get("histogramTimeseries", [])
good_pcts = []
if len(histogram_ts) >= 3:
good_densities = histogram_ts[0].get("densities", [])
for d in good_densities:
if d is None or str(d) == "NaN":
good_pcts.append(None)
else:
try:
good_pcts.append(round(float(d) * 100, 1))
except (ValueError, TypeError):
good_pcts.append(None)
# Extract needs_improvement (bin 1) and poor (bin 2) percentages
ni_pcts = []
poor_pcts = []
if len(histogram_ts) >= 3:
for bin_idx, target_list in [(1, ni_pcts), (2, poor_pcts)]:
bin_densities = histogram_ts[bin_idx].get("densities", [])
for d in bin_densities:
if d is None or str(d) == "NaN":
target_list.append(None)
else:
try:
target_list.append(round(float(d) * 100, 1))
except (ValueError, TypeError):
target_list.append(None)
result["metrics"][metric_name] = {
"label": thresholds["label"],
"unit": thresholds["unit"],
"p75_values": p75s,
"good_percentages": good_pcts,
"needs_improvement_percentages": ni_pcts,
"poor_percentages": poor_pcts,
"latest_p75": p75s[-1] if p75s and p75s[-1] is not None else None,
"good_threshold": thresholds["good"],
"poor_threshold": thresholds["poor"],
}
# Trend analysis
result["trends"] = detect_trends(result["metrics"])
return result
def detect_trends(metrics: dict) -> dict:
"""
Analyze p75 timeseries to detect trends.
Compares the average of the last 4 weeks to the average of the first 4 weeks.
Returns:
Dictionary mapping metric names to trend info:
direction (improving/stable/degrading), change_pct, latest, earliest.
"""
trends = {}
for metric_name, data in metrics.items():
p75s = data.get("p75_values", [])
valid = [v for v in p75s if v is not None]
if len(valid) < 8:
trends[metric_name] = {
"direction": "insufficient_data",
"label": data.get("label", metric_name),
}
continue
# First 4 valid vs last 4 valid
first_4 = valid[:4]
last_4 = valid[-4:]
avg_first = sum(first_4) / len(first_4)
avg_last = sum(last_4) / len(last_4)
if avg_first == 0:
change_pct = 0
else:
change_pct = ((avg_last - avg_first) / avg_first) * 100
# For CWV, lower is better (except CLS where lower is also better)
# So a negative change_pct means improvement
if abs(change_pct) < 5:
direction = "stable"
elif change_pct < 0:
direction = "improving"
else:
direction = "degrading"
trends[metric_name] = {
"direction": direction,
"change_pct": round(change_pct, 1),
"earliest_avg": round(avg_first, 3) if data.get("unit") == "" else round(avg_first),
"latest_avg": round(avg_last, 3) if data.get("unit") == "" else round(avg_last),
"label": data.get("label", metric_name),
"data_points": len(valid),
}
return trends
def main():
parser = argparse.ArgumentParser(
description="CrUX History API - Core Web Vitals trends over time"
)
parser.add_argument("url", help="URL or origin to analyze")
parser.add_argument(
"--form-factor",
choices=["PHONE", "DESKTOP", "TABLET"],
help="Filter by form factor",
)
parser.add_argument(
"--api-key",
help="Google API key (overrides config/env)",
)
parser.add_argument(
"--origin",
action="store_true",
help="Force origin-level query (strip path/query)",
)
parser.add_argument(
"--json", "-j",
action="store_true",
help="Output as JSON",
)
args = parser.parse_args()
api_key = args.api_key or get_api_key()
if not api_key:
print("Error: API key required. Use --api-key or configure GOOGLE_API_KEY.", file=sys.stderr)
sys.exit(1)
target = args.url
if args.origin:
parsed = urlparse(target)
target = f"{parsed.scheme}://{parsed.netloc}"
result = query_history(target, api_key, form_factor=args.form_factor)
if args.json:
print(json.dumps(result, indent=2))
else:
if result.get("error"):
print(f"Error: {result['error']}", file=sys.stderr)
sys.exit(1)
print(f"=== CrUX History ({result.get('form_factor', 'ALL')}) ===")
print(f"Target: {result.get('target')}")
periods = result.get("collection_periods", [])
if periods:
print(f"Range: {periods[0]['first']} to {periods[-1]['last']} ({len(periods)} weeks)")
print("\nTrend Analysis:")
for name, trend in result.get("trends", {}).items():
label = trend.get("label", name)
direction = trend.get("direction", "?")
if direction == "insufficient_data":
print(f" {label}: Insufficient data")
continue
arrow = {"improving": "IMPROVING", "stable": "STABLE", "degrading": "DEGRADING"}.get(direction, "?")
change = trend.get("change_pct", 0)
earliest = trend.get("earliest_avg")
latest = trend.get("latest_avg")
print(f" {label}: {arrow} ({change:+.1f}%) | {earliest} -> {latest}")
if __name__ == "__main__":
main()
#!/usr/bin/env python3
"""
GA4 Data API v1beta - organic traffic reporting.
Queries the Google Analytics Data API for organic search traffic,
top landing pages, and session metrics with channel filtering.
Usage:
python ga4_report.py --property 123456789
python ga4_report.py --property 123456789 --days 90 --report top-pages
python ga4_report.py --property 123456789 --report organic --json
"""
import argparse
import json
import sys
from datetime import datetime, timedelta
from typing import Optional
try:
from google.analytics.data_v1beta import BetaAnalyticsDataClient
from google.analytics.data_v1beta.types import (
DateRange,
Dimension,
Filter,
FilterExpression,
Metric,
OrderBy,
RunReportRequest,
)
except ImportError:
print(
"Error: google-analytics-data required. "
"Install with: pip install google-analytics-data",
file=sys.stderr,
)
sys.exit(1)
try:
from google_auth import get_oauth_credentials, load_config
except ImportError:
import os
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from google_auth import get_oauth_credentials, load_config
GA4_SCOPES = ["https://www.googleapis.com/auth/analytics.readonly"]
def _build_ga4_client():
"""Build the GA4 BetaAnalyticsDataClient."""
credentials = get_oauth_credentials(GA4_SCOPES)
if not credentials:
return None
try:
return BetaAnalyticsDataClient(credentials=credentials)
except Exception as e:
print(f"Error building GA4 client: {e}", file=sys.stderr)
return None
def _resolve_property(property_id: str) -> str:
"""Ensure property ID is in the correct format."""
if not property_id:
return ""
if property_id.startswith("properties/"):
return property_id
return f"properties/{property_id}"
def organic_traffic_report(
property_id: str,
days: int = 28,
limit: int = 100,
) -> dict:
"""
Generate organic traffic report from GA4.
Filters by sessionDefaultChannelGroup == "Organic Search" and returns
daily sessions, top landing pages, and key metrics.
Args:
property_id: GA4 property ID (numeric or 'properties/123456789').
days: Number of days to query (default: 28).
limit: Max rows (default: 100).
Returns:
Dictionary with daily_data, top_pages, totals, and quota usage.
"""
result = {
"property": property_id,
"report": "organic_traffic",
"date_range": None,
"totals": {},
"daily_data": [],
"top_pages": [],
"quota_tokens_used": None,
"error": None,
}
client = _build_ga4_client()
if not client:
result["error"] = (
"Could not build GA4 client. Ensure the service account has "
"Viewer access in GA4 Admin > Property Access Management."
)
return result
prop = _resolve_property(property_id)
start_date = (datetime.now() - timedelta(days=days)).strftime("%Y-%m-%d")
end_date = (datetime.now() - timedelta(days=1)).strftime("%Y-%m-%d")
result["date_range"] = {"start": start_date, "end": end_date}
# Daily organic sessions
try:
daily_request = RunReportRequest(
property=prop,
dimensions=[Dimension(name="date")],
metrics=[
Metric(name="sessions"),
Metric(name="totalUsers"),
Metric(name="screenPageViews"),
Metric(name="bounceRate"),
Metric(name="averageSessionDuration"),
Metric(name="engagementRate"),
],
date_ranges=[DateRange(start_date=start_date, end_date=end_date)],
dimension_filter=FilterExpression(
filter=Filter(
field_name="sessionDefaultChannelGroup",
string_filter=Filter.StringFilter(
match_type=Filter.StringFilter.MatchType.EXACT,
value="Organic Search",
),
)
),
order_bys=[OrderBy(dimension=OrderBy.DimensionOrderBy(dimension_name="date"))],
limit=days + 5,
return_property_quota=True,
)
daily_response = client.run_report(daily_request)
for row in daily_response.rows:
result["daily_data"].append({
"date": row.dimension_values[0].value,
"sessions": int(row.metric_values[0].value),
"users": int(row.metric_values[1].value),
"pageviews": int(row.metric_values[2].value),
"bounce_rate": round(float(row.metric_values[3].value) * 100, 1),
"avg_session_duration": round(float(row.metric_values[4].value), 1),
"engagement_rate": round(float(row.metric_values[5].value) * 100, 1),
})
# Quota info
if daily_response.property_quota:
pq = daily_response.property_quota
result["quota_tokens_used"] = {
"daily_consumed": pq.tokens_per_day.consumed if pq.tokens_per_day else None,
"daily_remaining": pq.tokens_per_day.remaining if pq.tokens_per_day else None,
"hourly_consumed": pq.tokens_per_hour.consumed if pq.tokens_per_hour else None,
"hourly_remaining": pq.tokens_per_hour.remaining if pq.tokens_per_hour else None,
}
except Exception as e:
error_str = str(e)
if "403" in error_str or "PERMISSION_DENIED" in error_str:
result["error"] = (
f"Permission denied for property '{property_id}'. "
"Add the service account email as Viewer in "
"GA4 Admin > Property Access Management."
)
elif "404" in error_str or "NOT_FOUND" in error_str:
result["error"] = (
f"Property '{property_id}' not found. "
"Verify the numeric property ID in GA4 Admin > Property Details."
)
else:
result["error"] = f"GA4 API error: {e}"
return result
# Top landing pages by organic sessions
try:
pages_request = RunReportRequest(
property=prop,
dimensions=[Dimension(name="landingPage")],
metrics=[
Metric(name="sessions"),
Metric(name="totalUsers"),
Metric(name="screenPageViews"),
Metric(name="bounceRate"),
Metric(name="engagementRate"),
],
date_ranges=[DateRange(start_date=start_date, end_date=end_date)],
dimension_filter=FilterExpression(
filter=Filter(
field_name="sessionDefaultChannelGroup",
string_filter=Filter.StringFilter(
match_type=Filter.StringFilter.MatchType.EXACT,
value="Organic Search",
),
)
),
order_bys=[
OrderBy(
metric=OrderBy.MetricOrderBy(metric_name="sessions"),
desc=True,
)
],
limit=limit,
)
pages_response = client.run_report(pages_request)
for row in pages_response.rows:
result["top_pages"].append({
"landing_page": row.dimension_values[0].value,
"sessions": int(row.metric_values[0].value),
"users": int(row.metric_values[1].value),
"pageviews": int(row.metric_values[2].value),
"bounce_rate": round(float(row.metric_values[3].value) * 100, 1),
"engagement_rate": round(float(row.metric_values[4].value) * 100, 1),
})
except Exception as e:
# Non-fatal: daily data succeeded, pages failed
result["pages_error"] = f"Error fetching top pages: {e}"
# Calculate totals
if result["daily_data"]:
total_sessions = sum(d["sessions"] for d in result["daily_data"])
total_users = sum(d["users"] for d in result["daily_data"])
total_pageviews = sum(d["pageviews"] for d in result["daily_data"])
result["totals"] = {
"sessions": total_sessions,
"users": total_users,
"pageviews": total_pageviews,
"avg_daily_sessions": round(total_sessions / len(result["daily_data"]), 1),
}
return result
def top_pages_report(
property_id: str,
days: int = 28,
limit: int = 50,
) -> dict:
"""
Get top organic landing pages from GA4.
Args:
property_id: GA4 property ID.
days: Number of days.
limit: Max pages to return.
Returns:
Dictionary with top pages ranked by organic sessions.
"""
report = organic_traffic_report(property_id, days, limit)
# Slim it down to just pages
return {
"property": property_id,
"report": "top_organic_pages",
"date_range": report.get("date_range"),
"pages": report.get("top_pages", []),
"total_organic_sessions": report.get("totals", {}).get("sessions", 0),
"quota_tokens_used": report.get("quota_tokens_used"),
"error": report.get("error"),
}
def device_breakdown(
property_id: str,
days: int = 28,
) -> dict:
"""
Organic sessions broken down by device category.
Args:
property_id: GA4 property ID.
days: Number of days.
Returns:
Dictionary with device breakdown data.
"""
result = {"property": property_id, "report": "device_breakdown", "devices": [], "error": None}
client = _build_ga4_client()
if not client:
result["error"] = "Could not build GA4 client."
return result
prop = _resolve_property(property_id)
start_date = (datetime.now() - timedelta(days=days)).strftime("%Y-%m-%d")
end_date = (datetime.now() - timedelta(days=1)).strftime("%Y-%m-%d")
result["date_range"] = {"start": start_date, "end": end_date}
try:
request = RunReportRequest(
property=prop,
dimensions=[Dimension(name="deviceCategory")],
metrics=[
Metric(name="sessions"),
Metric(name="totalUsers"),
Metric(name="bounceRate"),
Metric(name="engagementRate"),
],
date_ranges=[DateRange(start_date=start_date, end_date=end_date)],
dimension_filter=FilterExpression(
filter=Filter(
field_name="sessionDefaultChannelGroup",
string_filter=Filter.StringFilter(
match_type=Filter.StringFilter.MatchType.EXACT,
value="Organic Search",
),
)
),
order_bys=[OrderBy(metric=OrderBy.MetricOrderBy(metric_name="sessions"), desc=True)],
)
response = client.run_report(request)
for row in response.rows:
result["devices"].append({
"category": row.dimension_values[0].value,
"sessions": int(row.metric_values[0].value),
"users": int(row.metric_values[1].value),
"bounce_rate": round(float(row.metric_values[2].value) * 100, 1),
"engagement_rate": round(float(row.metric_values[3].value) * 100, 1),
})
except Exception as e:
result["error"] = f"GA4 device breakdown error: {e}"
return result
def country_breakdown(
property_id: str,
days: int = 28,
limit: int = 20,
) -> dict:
"""
Organic sessions broken down by country.
Args:
property_id: GA4 property ID.
days: Number of days.
limit: Max countries to return.
Returns:
Dictionary with country breakdown data.
"""
result = {"property": property_id, "report": "country_breakdown", "countries": [], "error": None}
client = _build_ga4_client()
if not client:
result["error"] = "Could not build GA4 client."
return result
prop = _resolve_property(property_id)
start_date = (datetime.now() - timedelta(days=days)).strftime("%Y-%m-%d")
end_date = (datetime.now() - timedelta(days=1)).strftime("%Y-%m-%d")
result["date_range"] = {"start": start_date, "end": end_date}
try:
request = RunReportRequest(
property=prop,
dimensions=[Dimension(name="country")],
metrics=[
Metric(name="sessions"),
Metric(name="totalUsers"),
],
date_ranges=[DateRange(start_date=start_date, end_date=end_date)],
dimension_filter=FilterExpression(
filter=Filter(
field_name="sessionDefaultChannelGroup",
string_filter=Filter.StringFilter(
match_type=Filter.StringFilter.MatchType.EXACT,
value="Organic Search",
),
)
),
order_bys=[OrderBy(metric=OrderBy.MetricOrderBy(metric_name="sessions"), desc=True)],
limit=limit,
)
response = client.run_report(request)
for row in response.rows:
result["countries"].append({
"country": row.dimension_values[0].value,
"sessions": int(row.metric_values[0].value),
"users": int(row.metric_values[1].value),
})
except Exception as e:
result["error"] = f"GA4 country breakdown error: {e}"
return result
def main():
parser = argparse.ArgumentParser(
description="GA4 Data API - organic traffic reporting"
)
parser.add_argument(
"--property", "-p",
help="GA4 property ID (numeric, e.g., 123456789). Uses config default if not specified.",
)
parser.add_argument("--days", "-d", type=int, default=28, help="Number of days (default: 28)")
parser.add_argument(
"--report", "-r",
choices=["organic", "top-pages", "device", "country"],
default="organic",
help="Report type (default: organic)",
)
parser.add_argument("--limit", type=int, default=50, help="Max rows (default: 50)")
parser.add_argument("--json", "-j", action="store_true", help="Output as JSON")
args = parser.parse_args()
# Resolve property
prop = args.property
if not prop:
config = load_config()
prop = config.get("ga4_property_id") or ""
# Strip 'properties/' prefix if present for consistency
if prop and prop.startswith("properties/"):
prop = prop[len("properties/"):]
if not prop:
print(
"Error: No GA4 property specified. Use --property or set ga4_property_id in config.",
file=sys.stderr,
)
sys.exit(1)
if args.report == "top-pages":
result = top_pages_report(prop, args.days, args.limit)
elif args.report == "device":
result = device_breakdown(prop, args.days)
elif args.report == "country":
result = country_breakdown(prop, args.days, args.limit)
else:
result = organic_traffic_report(prop, args.days, args.limit)
if result.get("error"):
print(f"Error: {result['error']}", file=sys.stderr)
if not args.json:
sys.exit(1)
if args.json:
print(json.dumps(result, indent=2, default=str))
else:
if args.report == "top-pages":
print(f"=== Top Organic Landing Pages ===")
print(f"Property: {prop} | Period: {result.get('date_range', {}).get('start')} to {result.get('date_range', {}).get('end')}")
print(f"Total organic sessions: {result.get('total_organic_sessions', 0):,}")
print()
for i, page in enumerate(result.get("pages", [])[:20], 1):
print(f" {i:2d}. {page['landing_page']}")
print(f" Sessions: {page['sessions']:,} | Users: {page['users']:,} | Bounce: {page['bounce_rate']}%")
else:
totals = result.get("totals", {})
print(f"=== GA4 Organic Traffic Report ===")
print(f"Property: {prop}")
dr = result.get("date_range", {})
print(f"Period: {dr.get('start')} to {dr.get('end')}")
print(f"\nSessions: {totals.get('sessions', 0):,} | Users: {totals.get('users', 0):,} | Pageviews: {totals.get('pageviews', 0):,}")
print(f"Avg Daily Sessions: {totals.get('avg_daily_sessions', 0):,.0f}")
quota = result.get("quota_tokens_used")
if quota and quota.get("daily_remaining") is not None:
print(f"\nQuota: {quota['daily_consumed']} tokens used / {quota['daily_remaining']} remaining (daily)")
pages = result.get("top_pages", [])
if pages:
print(f"\nTop {min(10, len(pages))} Organic Landing Pages:")
for i, page in enumerate(pages[:10], 1):
print(f" {i:2d}. {page['landing_page']} ({page['sessions']:,} sessions)")
if __name__ == "__main__":
main()
#!/usr/bin/env python3
"""
Google API credential management for Claude SEO.
Loads and validates credentials for Google Search Console, PageSpeed Insights,
CrUX, Indexing API, and GA4. Supports service accounts, OAuth web credentials
with token refresh, API keys, and environment variable fallbacks.
Usage:
python google_auth.py --check # Check all credentials
python google_auth.py --check gsc # Check specific service
python google_auth.py --check --json # JSON output
python google_auth.py --setup # Show setup instructions
python google_auth.py --tier # Show detected credential tier
python google_auth.py --auth --creds /path/to/client_secret.json # OAuth browser flow
"""
import argparse
import json
import os
import secrets
import sys
import tempfile
import time
from typing import Optional
CONFIG_PATH = os.path.expanduser("~/.config/claude-seo/google-api.json")
TOKEN_PATH = os.path.expanduser("~/.config/claude-seo/oauth-token.json")
# Service-to-scope mapping
SCOPES = {
"gsc_readonly": "https://www.googleapis.com/auth/webmasters.readonly",
"gsc_write": "https://www.googleapis.com/auth/webmasters",
"indexing": "https://www.googleapis.com/auth/indexing",
"ga4": "https://www.googleapis.com/auth/analytics.readonly",
}
# Which services need which auth type
SERVICE_AUTH = {
"psi": "api_key",
"crux": "api_key",
"crux_history": "api_key",
"gsc": "oauth_or_sa",
"indexing": "oauth_or_sa",
"ga4": "oauth_or_sa",
}
OAUTH_REDIRECT_URI = "http://127.0.0.1:8085"
def _write_secret_atomic(path: str, content: str) -> None:
"""Atomically write `content` to `path` with mode 0o600.
Uses tempfile in same dir + os.replace for atomicity (no partial writes
on crash). Sets restrictive file mode before writing payload.
"""
# Bare-filename safety: os.path.dirname returns "" if path has no dir
# component. Pass that to mkstemp(dir="") and it errors with FileNotFoundError.
parent = os.path.dirname(path) or "."
os.makedirs(parent, mode=0o700, exist_ok=True)
fd, tmp = tempfile.mkstemp(dir=parent, prefix=".tmp-")
try:
os.fchmod(fd, 0o600)
with os.fdopen(fd, "w") as f:
f.write(content)
os.replace(tmp, path)
os.chmod(path, 0o600) # belt-and-braces if file pre-existed
except Exception:
try:
os.unlink(tmp)
except OSError:
pass
raise
def _scopes_for(services: list = None) -> str:
"""Build OAuth scope string for the requested services.
Defaults to a read-only set so OAuth-without-flag grants minimal scopes.
"""
if services is None:
# Safer default: read-only scopes only.
services = ["gsc_readonly", "ga4"]
scope_urls = []
for s in services:
if s in SCOPES:
scope_urls.append(SCOPES[s])
else:
raise ValueError(f"Unknown scope service: {s}")
return " ".join(scope_urls)
# Human-readable service names
SERVICE_NAMES = {
"psi": "PageSpeed Insights v5",
"crux": "Chrome UX Report (CrUX) API",
"crux_history": "CrUX History API",
"gsc": "Google Search Console API",
"indexing": "Google Indexing API v3",
"ga4": "GA4 Data API v1beta",
}
def load_config() -> dict:
"""
Load configuration from config file with environment variable fallbacks.
Reads ~/.config/claude-seo/google-api.json first. Any missing fields
are filled from environment variables.
Returns:
Dictionary with keys: service_account_path, api_key,
default_property, ga4_property_id. Missing values are None.
"""
config = {
"service_account_path": None,
"api_key": None,
"default_property": None,
"ga4_property_id": None,
}
# Load from config file
if os.path.exists(CONFIG_PATH):
try:
with open(CONFIG_PATH, "r") as f:
file_config = json.load(f)
config.update({k: v for k, v in file_config.items() if v})
except (json.JSONDecodeError, IOError) as e:
print(f"Warning: Could not read config file: {e}", file=sys.stderr)
# Environment variable fallbacks
if not config["service_account_path"]:
config["service_account_path"] = os.environ.get("GOOGLE_APPLICATION_CREDENTIALS")
if not config["api_key"]:
config["api_key"] = os.environ.get("GOOGLE_API_KEY")
if not config["ga4_property_id"]:
config["ga4_property_id"] = os.environ.get("GA4_PROPERTY_ID")
if not config["default_property"]:
config["default_property"] = os.environ.get("GSC_PROPERTY")
return config
def get_service_account_credentials(scopes: list):
"""
Load Google service account credentials.
Args:
scopes: List of OAuth scope URLs.
Returns:
google.oauth2.service_account.Credentials object, or None on failure.
"""
try:
from google.oauth2 import service_account
except ImportError:
print(
"Error: google-auth library required. "
"Install with: pip install google-auth",
file=sys.stderr,
)
return None
config = load_config()
sa_path = config.get("service_account_path")
if not sa_path:
return None
sa_path = os.path.expanduser(sa_path)
if not os.path.exists(sa_path):
print(
f"Error: Service account file not found: {sa_path}",
file=sys.stderr,
)
return None
try:
credentials = service_account.Credentials.from_service_account_file(
sa_path, scopes=scopes
)
return credentials
except Exception as e:
print(f"Error loading service account: {e}", file=sys.stderr)
return None
def _load_oauth_client(creds_path: str) -> Optional[dict]:
"""Load OAuth client credentials from a client_secret JSON file."""
try:
with open(creds_path, "r") as f:
data = json.load(f)
return data.get("web", data.get("installed", {}))
except (json.JSONDecodeError, IOError) as e:
print(f"Error reading OAuth client file: {e}", file=sys.stderr)
return None
def _load_oauth_token() -> Optional[dict]:
"""Load saved OAuth token from TOKEN_PATH."""
if not os.path.exists(TOKEN_PATH):
return None
try:
with open(TOKEN_PATH, "r") as f:
return json.load(f)
except (json.JSONDecodeError, IOError):
return None
def _save_oauth_token(token_data: dict):
"""Save OAuth token to TOKEN_PATH with mode 0o600 and atomic write."""
_write_secret_atomic(TOKEN_PATH, json.dumps(token_data, indent=2))
def _refresh_oauth_token(client: dict, token_data: dict) -> Optional[dict]:
"""Refresh an expired OAuth token using the refresh_token."""
import urllib.parse
import urllib.request
if not token_data.get("refresh_token"):
return None
params = urllib.parse.urlencode({
"client_id": client["client_id"],
"client_secret": client["client_secret"],
"refresh_token": token_data["refresh_token"],
"grant_type": "refresh_token",
}).encode()
try:
req = urllib.request.Request(client.get("token_uri", "https://oauth2.googleapis.com/token"), data=params)
with urllib.request.urlopen(req) as resp:
new_data = json.loads(resp.read())
token_data["access_token"] = new_data["access_token"]
token_data["expires_at"] = time.time() + new_data.get("expires_in", 3600)
if "refresh_token" in new_data: # Google now sometimes rotates these
token_data["refresh_token"] = new_data["refresh_token"]
# AUTH-001 (v1.9.1): strip client_secret on every save so older
# token files migrate forward the first time they're refreshed.
token_data.pop("client_secret", None)
_save_oauth_token(token_data)
return token_data
except Exception as e:
print(f"Error refreshing OAuth token: {e}", file=sys.stderr)
return None
def get_oauth_credentials(scopes: list):
"""
Get OAuth credentials from saved token, refreshing if needed.
Falls back to service account if no OAuth token is available.
Args:
scopes: List of OAuth scope URLs (used for service account fallback).
Returns:
google.oauth2.credentials.Credentials or service_account.Credentials, or None.
"""
config = load_config()
# Try OAuth token first
token_data = _load_oauth_token()
if token_data and token_data.get("access_token"):
# Check if token needs refresh
if time.time() > token_data.get("expires_at", 0) - 60:
oauth_creds_path = config.get("oauth_client_path")
if oauth_creds_path:
client = _load_oauth_client(os.path.expanduser(oauth_creds_path))
if client:
token_data = _refresh_oauth_token(client, token_data)
if not token_data:
print("OAuth token refresh failed. Re-run --auth.", file=sys.stderr)
return get_service_account_credentials(scopes)
if token_data and token_data.get("access_token"):
try:
from google.oauth2.credentials import Credentials
# AUTH-001 (v1.9.1): client_secret is no longer stored in
# the token file. Re-read from config["oauth_client_path"]
# so the long-lived app credential stays in its own
# 0o600 file (and is referenced, not duplicated).
# Backwards-compat: legacy token files still containing
# client_secret are honored (token_data.get fallback)
# until v1.10.0 makes oauth_client_path mandatory.
client_secret = None
oauth_creds_path = config.get("oauth_client_path")
if oauth_creds_path:
client = _load_oauth_client(os.path.expanduser(oauth_creds_path))
if client:
client_secret = client.get("client_secret")
if client_secret is None:
client_secret = token_data.get("client_secret") # legacy compat
return Credentials(
token=token_data["access_token"],
refresh_token=token_data.get("refresh_token"),
token_uri="https://oauth2.googleapis.com/token",
client_id=token_data.get("client_id"),
client_secret=client_secret,
)
except ImportError:
print("Error: google-auth required. Install with: pip install google-auth", file=sys.stderr)
# Fall back to service account
return get_service_account_credentials(scopes)
def run_oauth_flow(creds_path: str, services: list = None):
"""
Run OAuth browser-based authentication flow.
Opens a browser for consent, captures the auth code via local HTTP server,
exchanges for tokens, and saves them.
Args:
creds_path: Path to the OAuth client_secret JSON file.
services: Optional list of scope service keys (see SCOPES). Defaults to
the read-only set built by `_scopes_for(None)`.
"""
import http.server
import urllib.parse
import urllib.request
import webbrowser
client = _load_oauth_client(creds_path)
if not client:
print("Error: Could not load OAuth client credentials.", file=sys.stderr)
sys.exit(1)
state_token = secrets.token_urlsafe(32)
scopes_str = _scopes_for(services)
auth_params = {
"client_id": client["client_id"],
"redirect_uri": OAUTH_REDIRECT_URI,
"response_type": "code",
"scope": scopes_str,
"access_type": "offline",
"prompt": "consent",
"state": state_token,
"include_granted_scopes": "true",
}
auth_url = (
f"{client.get('auth_uri', 'https://accounts.google.com/o/oauth2/auth')}"
f"?{urllib.parse.urlencode(auth_params)}"
)
auth_code = [None]
class Handler(http.server.BaseHTTPRequestHandler):
def do_GET(self):
params = urllib.parse.parse_qs(urllib.parse.urlparse(self.path).query)
received_state = params.get("state", [""])[0]
if received_state != state_token:
self.send_response(403)
self.send_header("Content-Type", "text/plain")
self.end_headers()
self.wfile.write(b"State mismatch - possible CSRF. Aborted.")
return
if "code" in params:
auth_code[0] = params["code"][0]
self.send_response(200)
self.send_header("Content-Type", "text/html")
self.end_headers()
self.wfile.write(b"<html><body><h1>Authorization complete.</h1>You can close this tab.</body></html>")
else:
self.send_response(400)
self.end_headers()
def log_message(self, *a):
pass
server = http.server.HTTPServer(("127.0.0.1", 8085), Handler)
server.timeout = 300
print(f"\nOpen this URL in your browser:\n\n{auth_url}\n")
print("Waiting up to 5 minutes for authentication...")
try:
webbrowser.open(auth_url)
except Exception:
pass
server.handle_request()
server.server_close()
if not auth_code[0]:
print("\nAuthentication failed or timed out.", file=sys.stderr)
print("If the browser showed '127.0.0.1 refused to connect', copy the full URL")
print("from the browser address bar and run:")
print(f" python scripts/google_auth.py --exchange --creds {creds_path} --code 'THE_CODE'")
sys.exit(1)
# Exchange code for tokens
_exchange_code(client, auth_code[0])
def _exchange_code(client: dict, code: str):
"""Exchange an authorization code for tokens."""
import urllib.parse
import urllib.request
params = urllib.parse.urlencode({
"code": code,
"client_id": client["client_id"],
"client_secret": client["client_secret"],
"redirect_uri": OAUTH_REDIRECT_URI,
"grant_type": "authorization_code",
}).encode()
try:
req = urllib.request.Request(
client.get("token_uri", "https://oauth2.googleapis.com/token"), data=params
)
with urllib.request.urlopen(req) as resp:
token_data = json.loads(resp.read())
token_data["expires_at"] = time.time() + token_data.get("expires_in", 3600)
token_data["client_id"] = client["client_id"]
# AUTH-001 (v1.9.1): client_secret is NO LONGER stored in the token
# file. Co-locating the long-lived app credential with the
# short-lived access token expands blast radius if the token file
# leaks. Refresh paths re-read client_secret from
# config["oauth_client_path"] instead. See get_oauth_credentials.
token_data.pop("client_secret", None)
_save_oauth_token(token_data)
print("OAuth token saved successfully!")
# Also save the OAuth client path to config
config = load_config()
# Don't overwrite existing config, just suggest
print(f"\nToken saved to: {TOKEN_PATH}")
except Exception as e:
print(f"Error exchanging authorization code: {e}", file=sys.stderr)
sys.exit(1)
def get_api_key() -> Optional[str]:
"""
Get the Google API key from config or environment.
Returns:
API key string, or None if not configured.
"""
config = load_config()
return config.get("api_key")
def build_service(api_name: str, version: str, scopes: list):
"""
Build a Google API discovery service client.
Args:
api_name: API name (e.g., 'searchconsole', 'indexing', 'pagespeedonline').
version: API version (e.g., 'v1', 'v3', 'v5').
scopes: OAuth scopes needed.
Returns:
googleapiclient.discovery.Resource object, or None on failure.
"""
try:
from googleapiclient.discovery import build
except ImportError:
print(
"Error: google-api-python-client required. "
"Install with: pip install google-api-python-client",
file=sys.stderr,
)
return None
credentials = get_oauth_credentials(scopes)
if not credentials:
return None
try:
service = build(api_name, version, credentials=credentials)
return service
except Exception as e:
print(f"Error building {api_name} service: {e}", file=sys.stderr)
return None
def check_credentials(service: str) -> dict:
"""
Validate credentials for a specific Google API service.
Args:
service: One of 'psi', 'crux', 'crux_history', 'gsc', 'indexing', 'ga4'.
Returns:
Dictionary with:
- available: bool
- method: 'api_key' or 'service_account'
- service: service name
- error: error message or None
"""
result = {
"available": False,
"method": SERVICE_AUTH.get(service, "unknown"),
"service": SERVICE_NAMES.get(service, service),
"error": None,
}
config = load_config()
if SERVICE_AUTH.get(service) == "api_key":
api_key = config.get("api_key")
if api_key:
result["available"] = True
else:
result["error"] = (
"No API key found. Set GOOGLE_API_KEY environment variable "
f"or add 'api_key' to {CONFIG_PATH}"
)
elif SERVICE_AUTH.get(service) == "oauth_or_sa":
# Check OAuth token first
token_data = _load_oauth_token()
if token_data and token_data.get("access_token"):
result["available"] = True
result["method"] = "oauth_token"
expired = time.time() > token_data.get("expires_at", 0) - 60
if expired and token_data.get("refresh_token"):
result["note"] = "Token expired but refresh_token available (will auto-refresh)"
elif expired:
result["available"] = False
result["error"] = "OAuth token expired and no refresh_token. Re-run --auth."
else:
# Fall back to service account
sa_path = config.get("service_account_path")
if not sa_path:
result["error"] = (
"No OAuth token or service account found. Either:\n"
" 1. Run: python scripts/google_auth.py --auth --creds /path/to/client_secret.json\n"
f" 2. Or add 'service_account_path' to {CONFIG_PATH}"
)
else:
sa_path = os.path.expanduser(sa_path)
if not os.path.exists(sa_path):
result["error"] = f"Service account file not found: {sa_path}"
else:
try:
with open(sa_path, "r") as f:
sa_data = json.load(f)
if "client_email" not in sa_data or "private_key" not in sa_data:
result["error"] = "Service account JSON missing required fields (client_email, private_key)"
else:
result["available"] = True
result["method"] = "service_account"
result["client_email"] = sa_data.get("client_email")
except (json.JSONDecodeError, IOError) as e:
result["error"] = f"Invalid service account file: {e}"
# GA4 also needs property ID
if service == "ga4" and result["available"]:
ga4_id = config.get("ga4_property_id")
if not ga4_id:
result["available"] = False
result["error"] = (
"Credentials found but no GA4 property ID configured. "
f"Set GA4_PROPERTY_ID or add 'ga4_property_id' to {CONFIG_PATH}"
)
else:
result["error"] = f"Unknown service: {service}"
return result
def detect_tier() -> dict:
"""
Detect the credential tier available.
Returns:
Dictionary with:
- tier: 0, 1, or 2
- description: human-readable tier description
- capabilities: list of available API groups
- missing: what's needed for the next tier
"""
config = load_config()
has_api_key = bool(config.get("api_key"))
has_authenticated = False
has_ga4 = False
auth_method = None
# Check OAuth token
token_data = _load_oauth_token()
if token_data and token_data.get("access_token"):
has_authenticated = True
auth_method = "oauth_token"
# Check service account
if not has_authenticated:
sa_path = config.get("service_account_path")
if sa_path:
sa_path = os.path.expanduser(sa_path)
if os.path.exists(sa_path):
try:
with open(sa_path, "r") as f:
sa_data = json.load(f)
if "client_email" in sa_data and "private_key" in sa_data:
has_authenticated = True
auth_method = "service_account"
except (json.JSONDecodeError, IOError):
pass
if has_authenticated and config.get("ga4_property_id"):
has_ga4 = True
if has_ga4:
return {
"tier": 2,
"description": "Full (API key + Service Account + GA4)",
"capabilities": [
"PageSpeed Insights", "CrUX", "CrUX History",
"Search Console", "URL Inspection", "Sitemaps",
"Indexing API", "GA4 Organic Traffic",
],
"missing": None,
}
elif has_authenticated:
return {
"tier": 1,
"description": "Authenticated (API key + OAuth/Service Account)",
"capabilities": [
"PageSpeed Insights", "CrUX", "CrUX History",
"Search Console", "URL Inspection", "Sitemaps",
"Indexing API",
],
"missing": "Add 'ga4_property_id' to unlock GA4 organic traffic reports",
}
elif has_api_key:
return {
"tier": 0,
"description": "API Key Only",
"capabilities": [
"PageSpeed Insights", "CrUX", "CrUX History",
],
"missing": "Add a service account to unlock Search Console, URL Inspection, and Indexing API",
}
else:
return {
"tier": -1,
"description": "No credentials configured",
"capabilities": [],
"missing": (
f"Create config at {CONFIG_PATH} with at minimum an 'api_key' field. "
"Run with --setup for full instructions."
),
}
def print_setup_instructions():
"""Print step-by-step setup instructions."""
print("""
Google SEO API Setup Instructions
=================================
1. CREATE A GOOGLE CLOUD PROJECT
- Go to https://console.cloud.google.com
- Create a new project (or select existing)
- Note the project ID
2. ENABLE APIs
In API Library (APIs & Services > Library), enable:
- Google Search Console API
- PageSpeed Insights API
- Chrome UX Report API
- Web Search Indexing API (for Indexing API)
- Google Analytics Data API (for GA4)
3. CREATE AN API KEY (for PSI, CrUX - free, no service account needed)
- APIs & Services > Credentials > Create Credentials > API key
- Restrict to: PageSpeed Insights API, Chrome UX Report API
4. CREATE A SERVICE ACCOUNT (for GSC, Indexing API, GA4)
- IAM & Admin > Service Accounts > Create Service Account
- Download JSON key file, store securely
5. GRANT ACCESS
- Search Console: Settings > Users and permissions > Add user
Paste the service account client_email, set as Owner (for Indexing API) or Full (read-only)
- GA4: Admin > Property Access Management > Add
Paste email, set Viewer role
6. CREATE CONFIG FILE
mkdir -p ~/.config/claude-seo
Save to ~/.config/claude-seo/google-api.json:
{
"service_account_path": "/path/to/service_account.json",
"api_key": "AIzaSy...",
"default_property": "sc-domain:example.com",
"ga4_property_id": "properties/123456789"
}
7. VERIFY
python scripts/google_auth.py --check
ENVIRONMENT VARIABLE ALTERNATIVES:
GOOGLE_API_KEY - API key
GOOGLE_APPLICATION_CREDENTIALS - Path to service account JSON
GA4_PROPERTY_ID - GA4 property ID (e.g., properties/123456789)
GSC_PROPERTY - Default Search Console property
""")
def main():
parser = argparse.ArgumentParser(
description="Google API credential management for Claude SEO"
)
parser.add_argument(
"--check",
nargs="?",
const="all",
metavar="SERVICE",
help="Check credentials. Optionally specify service: psi, crux, gsc, indexing, ga4",
)
parser.add_argument(
"--setup",
action="store_true",
help="Show setup instructions",
)
parser.add_argument(
"--tier",
action="store_true",
help="Show detected credential tier",
)
parser.add_argument(
"--json",
action="store_true",
help="Output as JSON",
)
parser.add_argument(
"--auth",
action="store_true",
help="Run OAuth browser-based authentication flow",
)
parser.add_argument(
"--exchange",
action="store_true",
help="Manually exchange an auth code for tokens",
)
parser.add_argument(
"--creds",
help="Path to OAuth client_secret JSON file (for --auth and --exchange)",
)
parser.add_argument(
"--code",
help="Authorization code to exchange (for --exchange)",
)
parser.add_argument(
"--scopes",
help=(
"Comma-separated scope service keys for --auth (e.g. "
"'gsc_readonly,ga4' or 'indexing,gsc_write'). Defaults to a "
"read-only set: gsc_readonly,ga4."
),
)
args = parser.parse_args()
if args.auth:
if not args.creds:
print("Error: --creds is required with --auth", file=sys.stderr)
sys.exit(1)
services = None
if args.scopes:
services = [s.strip() for s in args.scopes.split(",") if s.strip()]
try:
run_oauth_flow(args.creds, services=services)
except ValueError as e:
# _scopes_for raises ValueError for unknown service keys.
# Surface a clean error instead of a stack trace.
print(f"Error: {e}", file=sys.stderr)
print(
f"Valid scope keys: {', '.join(sorted(SCOPES.keys()))}",
file=sys.stderr,
)
sys.exit(2)
return
if args.exchange:
if not args.creds or not args.code:
print("Error: --creds and --code are required with --exchange", file=sys.stderr)
sys.exit(1)
client = _load_oauth_client(args.creds)
if client:
_exchange_code(client, args.code)
return
if args.setup:
print_setup_instructions()
return
if args.tier:
tier_info = detect_tier()
if args.json:
print(json.dumps(tier_info, indent=2))
else:
print(f"Credential Tier: {tier_info['tier']} - {tier_info['description']}")
if tier_info["capabilities"]:
print(f"Available APIs: {', '.join(tier_info['capabilities'])}")
if tier_info["missing"]:
print(f"Next tier: {tier_info['missing']}")
return
if args.check:
services = (
list(SERVICE_AUTH.keys())
if args.check == "all"
else [args.check]
)
results = {}
for svc in services:
if svc not in SERVICE_AUTH:
results[svc] = {"available": False, "error": f"Unknown service: {svc}"}
continue
results[svc] = check_credentials(svc)
if args.json:
tier_info = detect_tier()
output = {"tier": tier_info, "services": results}
print(json.dumps(output, indent=2))
else:
tier_info = detect_tier()
print(f"Credential Tier: {tier_info['tier']} - {tier_info['description']}")
print()
for svc, result in results.items():
status = "OK" if result["available"] else "MISSING"
print(f" [{status}] {result.get('service', svc)}")
if result.get("error"):
print(f" {result['error']}")
if result.get("client_email"):
print(f" Service account: {result['client_email']}")
print()
if tier_info["missing"]:
print(f"Tip: {tier_info['missing']}")
return
# Default: show tier
tier_info = detect_tier()
if args.json:
print(json.dumps(tier_info, indent=2))
else:
print(f"Credential Tier: {tier_info['tier']} - {tier_info['description']}")
if tier_info["missing"]:
print(f"Run --setup for configuration instructions.")
if __name__ == "__main__":
main()
#!/usr/bin/env python3
"""
Google SEO Report Generator - Professional PDF/HTML reports from API data.
Consumes JSON output from seo-google scripts and generates formatted reports
with charts, analytics, and actionable recommendations.
Usage:
python google_report.py --type cwv-audit --data cwv-data.json --domain example.com
python google_report.py --type gsc-performance --data gsc-data.json --domain example.com
python google_report.py --type indexation --data inspect-data.json --domain example.com
python google_report.py --type full --data full-data.json --domain example.com
cat data.json | python google_report.py --type cwv-audit --domain example.com
"""
import argparse
import json
import os
import sys
from datetime import datetime
from pathlib import Path
from typing import Optional
try:
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
import matplotlib.patches as mpatches
import numpy as np
except ImportError:
print("Error: matplotlib required. Install with: pip install matplotlib", file=sys.stderr)
sys.exit(1)
try:
from weasyprint import HTML
except ImportError:
print("Error: weasyprint required. Install with: pip install weasyprint", file=sys.stderr)
sys.exit(1)
# ─── Brand Colors ────────────────────────────────────────────────────────────
BRAND = {
"primary": "#1a56db",
"secondary": "#6366f1",
"accent": "#06b6d4",
"success": "#10b981",
"warning": "#f59e0b",
"danger": "#ef4444",
"dark": "#1e293b",
"light_bg": "#f8fafc",
"grid": "#e2e8f0",
"muted": "#94a3b8",
}
def _score_color(score):
if score >= 90:
return BRAND["success"]
elif score >= 50:
return BRAND["warning"]
return BRAND["danger"]
def _rating_color(rating):
r = str(rating).lower().replace("-", "_").replace(" ", "_")
if r in ("good", "pass", "fast"):
return BRAND["success"]
elif r in ("needs_improvement", "needs-improvement", "average", "warn"):
return BRAND["warning"]
return BRAND["danger"]
# ─── Chart Setup ─────────────────────────────────────────────────────────────
def _setup_matplotlib():
plt.rcParams.update({
"font.family": "sans-serif",
"font.sans-serif": ["DejaVu Sans", "Arial", "Helvetica"],
"font.size": 11,
"axes.titlesize": 14,
"axes.titleweight": "bold",
"axes.labelsize": 11,
"axes.facecolor": "white",
"figure.facecolor": "white",
"axes.grid": False,
"axes.spines.top": False,
"axes.spines.right": False,
})
_setup_matplotlib()
# ─── Chart Functions ─────────────────────────────────────────────────────────
def chart_lighthouse_gauges(data: dict, output_dir: Path) -> str:
"""Generate 2x2 Lighthouse score gauges."""
scores = data.get("lighthouse_scores", {})
if not scores:
return ""
fig, axes = plt.subplots(2, 2, figsize=(8, 6), subplot_kw={"projection": "polar"})
categories = [
("performance", "Performance"),
("accessibility", "Accessibility"),
("best-practices", "Best Practices"),
("seo", "SEO"),
]
for ax, (key, label) in zip(axes.flat, categories):
score = scores.get(key, 0)
theta_bg = np.linspace(np.pi, 0, 100)
theta_fill = np.linspace(np.pi, np.pi - (score / 100) * np.pi, 100)
ax.plot(theta_bg, [1] * 100, linewidth=16, color="#e2e8f0", solid_capstyle="round")
ax.plot(theta_fill, [1] * 100, linewidth=16, color=_score_color(score), solid_capstyle="round")
ax.text(np.pi / 2, 0.35, f"{score}", ha="center", va="center",
fontsize=28, fontweight="bold", color=BRAND["dark"])
ax.text(np.pi / 2, -0.05, label, ha="center", va="center",
fontsize=10, color=BRAND["muted"])
ax.set_ylim(0, 1.3)
ax.set_rticks([])
ax.set_thetagrids([])
ax.spines["polar"].set_visible(False)
plt.tight_layout(pad=2)
path = output_dir / "lighthouse_gauges.png"
plt.savefig(path, dpi=200, bbox_inches="tight", facecolor="white")
plt.close()
return str(path)
def chart_cwv_distributions(data: dict, output_dir: Path) -> str:
"""Generate stacked horizontal bars for CWV metric distributions."""
crux = data.get("crux", {})
metrics = crux.get("metrics", {})
if not metrics:
return ""
cwv_order = [
"largest_contentful_paint", "interaction_to_next_paint",
"cumulative_layout_shift", "first_contentful_paint",
"experimental_time_to_first_byte",
]
labels, goods, nis, poors = [], [], [], []
for name in cwv_order:
m = metrics.get(name)
if not m or "distribution" not in m:
continue
d = m["distribution"]
labels.append(m.get("label", name))
goods.append(d.get("good", 0))
nis.append(d.get("needs_improvement", 0))
poors.append(d.get("poor", 0))
if not labels:
return ""
fig, ax = plt.subplots(figsize=(8, max(2.5, len(labels) * 0.7)))
y = range(len(labels))
ax.barh(y, goods, color=BRAND["success"], label="Good", height=0.5)
ax.barh(y, nis, left=goods, color=BRAND["warning"], label="Needs Improvement", height=0.5)
left2 = [g + n for g, n in zip(goods, nis)]
ax.barh(y, poors, left=left2, color=BRAND["danger"], label="Poor", height=0.5)
ax.set_yticks(y)
ax.set_yticklabels(labels)
ax.set_xlim(0, 100)
ax.set_xlabel("% of page loads")
ax.legend(loc="lower right", fontsize=9)
ax.invert_yaxis()
for i, (g, n, p) in enumerate(zip(goods, nis, poors)):
if g > 10:
ax.text(g / 2, i, f"{g:.0f}%", ha="center", va="center", fontsize=8, color="white", fontweight="bold")
if n > 10:
ax.text(g + n / 2, i, f"{n:.0f}%", ha="center", va="center", fontsize=8, color="white", fontweight="bold")
if p > 10:
ax.text(g + n + p / 2, i, f"{p:.0f}%", ha="center", va="center", fontsize=8, color="white", fontweight="bold")
plt.tight_layout()
path = output_dir / "cwv_distributions.png"
plt.savefig(path, dpi=200, bbox_inches="tight", facecolor="white")
plt.close()
return str(path)
def chart_cwv_timeline(data: dict, output_dir: Path) -> str:
"""Generate CWV timeline chart from CrUX History data."""
metrics = data.get("metrics", {})
periods = data.get("collection_periods", [])
if not metrics or not periods:
return ""
cwv_metrics = ["largest_contentful_paint", "interaction_to_next_paint", "cumulative_layout_shift"]
available = [m for m in cwv_metrics if m in metrics]
if not available:
return ""
fig, axes = plt.subplots(len(available), 1, figsize=(10, 3 * len(available)), sharex=True)
if len(available) == 1:
axes = [axes]
x_labels = [p.get("last", "")[-5:] for p in periods] # MM-DD format
x = range(len(x_labels))
for ax, metric_name in zip(axes, available):
m = metrics[metric_name]
p75s = m.get("p75_values", [])
label = m.get("label", metric_name)
good_t = m.get("good_threshold", 0)
poor_t = m.get("poor_threshold", 0)
valid_x = [i for i, v in enumerate(p75s) if v is not None]
valid_y = [v for v in p75s if v is not None]
if not valid_y:
continue
# Threshold bands
if good_t and poor_t:
ax.axhspan(0, good_t, alpha=0.1, color=BRAND["success"])
ax.axhspan(good_t, poor_t, alpha=0.1, color=BRAND["warning"])
ax.axhline(y=good_t, color=BRAND["success"], linestyle="--", alpha=0.5, linewidth=1)
ax.axhline(y=poor_t, color=BRAND["danger"], linestyle="--", alpha=0.5, linewidth=1)
ax.plot(valid_x, valid_y, color=BRAND["primary"], linewidth=2, marker="o", markersize=3)
ax.fill_between(valid_x, valid_y, alpha=0.1, color=BRAND["primary"])
unit = m.get("unit", "")
ax.set_ylabel(f"{label} (p75{unit})")
ax.set_title(label, fontsize=12, fontweight="bold")
if x_labels:
step = max(1, len(x_labels) // 8)
axes[-1].set_xticks(range(0, len(x_labels), step))
axes[-1].set_xticklabels([x_labels[i] for i in range(0, len(x_labels), step)], rotation=45, fontsize=8)
plt.tight_layout()
path = output_dir / "cwv_timeline.png"
plt.savefig(path, dpi=200, bbox_inches="tight", facecolor="white")
plt.close()
return str(path)
def chart_top_queries(data: dict, output_dir: Path) -> str:
"""Generate horizontal bar chart of top queries by clicks."""
rows = data.get("rows", [])
if not rows:
return ""
top = sorted(rows, key=lambda r: r.get("clicks", 0), reverse=True)[:15]
if not top:
return ""
labels = [r.get("query", r.get("keys", ["?"])[0])[:40] for r in top]
clicks = [r.get("clicks", 0) for r in top]
fig, ax = plt.subplots(figsize=(8, max(3, len(labels) * 0.4)))
y = range(len(labels))
bars = ax.barh(y, clicks, color=BRAND["primary"], height=0.6)
ax.set_yticks(y)
ax.set_yticklabels(labels, fontsize=9)
ax.set_xlabel("Clicks")
ax.invert_yaxis()
for bar, val in zip(bars, clicks):
if val > 0:
ax.text(bar.get_width() + max(clicks) * 0.02, bar.get_y() + bar.get_height() / 2,
str(val), va="center", fontsize=8, color=BRAND["dark"])
plt.tight_layout()
path = output_dir / "top_queries.png"
plt.savefig(path, dpi=200, bbox_inches="tight", facecolor="white")
plt.close()
return str(path)
def chart_index_status(data: dict, output_dir: Path) -> str:
"""Generate donut chart for URL inspection results."""
summary = data.get("summary", {})
if not summary:
return ""
labels, sizes, colors = [], [], []
for key, label, color in [
("pass", "Indexed", BRAND["success"]),
("fail", "Not Indexed", BRAND["danger"]),
("neutral", "Neutral", BRAND["grid"]),
("error", "Error", BRAND["muted"]),
]:
val = summary.get(key, 0)
if val > 0:
labels.append(f"{label} ({val})")
sizes.append(val)
colors.append(color)
if not sizes:
return ""
fig, ax = plt.subplots(figsize=(5, 4))
wedges, texts, autotexts = ax.pie(
sizes, labels=labels, colors=colors, autopct="%1.0f%%",
startangle=90, pctdistance=0.75, textprops={"fontsize": 9},
)
centre = plt.Circle((0, 0), 0.50, fc="white")
ax.add_artist(centre)
total = sum(sizes)
ax.text(0, 0, f"{total}\nURLs", ha="center", va="center",
fontsize=16, fontweight="bold", color=BRAND["dark"])
plt.tight_layout()
path = output_dir / "index_status.png"
plt.savefig(path, dpi=200, bbox_inches="tight", facecolor="white")
plt.close()
return str(path)
# ─── CSS Template ────────────────────────────────────────────────────────────
def _base_css(domain: str) -> str:
"""Battle-tested A4 report CSS extracted from generate_pdf.py."""
return f"""
@page {{ size: A4; margin: 22mm 18mm 25mm 18mm;
@bottom-center {{ content: counter(page); font-size: 9pt; color: #94a3b8; font-family: 'DejaVu Sans', Arial, sans-serif; }}
@bottom-right {{ content: "{domain} Google SEO Report"; font-size: 8pt; color: #cbd5e1; font-family: 'DejaVu Sans', Arial, sans-serif; }}
}}
@page :first {{ margin: 0; @bottom-center {{ content: none; }} @bottom-right {{ content: none; }} }}
* {{ box-sizing: border-box; margin: 0; padding: 0; }}
body {{ font-family: 'DejaVu Sans', Arial, Helvetica, sans-serif; font-size: 10pt; line-height: 1.55; color: #1e293b; background: white; }}
.title-page {{ page: first; width: 210mm; height: 297mm; background: linear-gradient(135deg, #0f172a 0%, #1e3a5f 50%, #1a56db 100%); display: flex; flex-direction: column; justify-content: center; align-items: center; text-align: center; color: white; position: relative; padding: 40mm 25mm; }}
.title-page .badge {{ background: rgba(255,255,255,0.12); border: 1px solid rgba(255,255,255,0.2); border-radius: 20px; padding: 6px 18px; font-size: 10pt; letter-spacing: 2px; text-transform: uppercase; margin-bottom: 20mm; color: #93c5fd; }}
.title-page h1 {{ font-size: 30pt; font-weight: bold; margin-bottom: 6mm; letter-spacing: -0.5px; line-height: 1.2; }}
.title-page .subtitle {{ font-size: 16pt; color: #93c5fd; margin-bottom: 12mm; font-weight: 300; }}
.title-page .url {{ font-size: 14pt; color: #60a5fa; margin-bottom: 20mm; padding: 5mm 10mm; border: 1px solid rgba(96, 165, 250, 0.3); border-radius: 8px; background: rgba(96, 165, 250, 0.08); }}
.title-page .score-box {{ background: rgba(255,255,255,0.1); border: 2px solid rgba(255,255,255,0.2); border-radius: 16px; padding: 8mm 15mm; margin-bottom: 15mm; }}
.title-page .score-number {{ font-size: 48pt; font-weight: bold; color: #fbbf24; line-height: 1; }}
.title-page .score-label {{ font-size: 11pt; color: #93c5fd; margin-top: 2mm; }}
.title-page .meta {{ font-size: 10pt; color: #94a3b8; margin-top: 10mm; }}
div.section {{ page-break-before: always; }}
.section-header {{ background: #f8fafc; border-left: 4px solid #1a56db; padding: 5mm 6mm; margin-bottom: 6mm; page-break-after: avoid; }}
.section-header h2 {{ font-size: 16pt; color: #0f172a; margin-bottom: 1mm; }}
.section-header .section-score {{ font-size: 12pt; font-weight: bold; float: right; margin-top: -6mm; }}
h3 {{ font-size: 12pt; color: #1a56db; margin-top: 6mm; margin-bottom: 3mm; padding-bottom: 1.5mm; border-bottom: 1px solid #e2e8f0; page-break-after: avoid; }}
h4 {{ font-size: 10.5pt; color: #334155; margin-top: 4mm; margin-bottom: 2mm; page-break-after: avoid; }}
p {{ margin-bottom: 3mm; color: #334155; }}
.highlight {{ background: #fef3c7; border-left: 3px solid #f59e0b; padding: 3mm 4mm; margin: 4mm 0; font-size: 9.5pt; page-break-inside: avoid; }}
.critical-box {{ background: #fef2f2; border-left: 3px solid #ef4444; padding: 3mm 4mm; margin: 4mm 0; font-size: 9.5pt; page-break-inside: avoid; }}
.success-box {{ background: #f0fdf4; border-left: 3px solid #10b981; padding: 3mm 4mm; margin: 4mm 0; font-size: 9.5pt; page-break-inside: avoid; }}
table {{ width: 100%; border-collapse: collapse; margin: 4mm 0 6mm 0; font-size: 9pt; page-break-inside: avoid; }}
thead th {{ background: #f1f5f9; color: #0f172a; font-weight: bold; padding: 2.5mm 3mm; text-align: left; border-bottom: 2px solid #cbd5e1; font-size: 9pt; }}
tbody td {{ padding: 2.5mm 3mm; border-bottom: 1px solid #f1f5f9; vertical-align: top; }}
tbody tr:nth-child(even) {{ background: #fafbfc; }}
.status-pass {{ color: #10b981; font-weight: bold; }}
.status-fail {{ color: #ef4444; font-weight: bold; }}
.status-warn {{ color: #f59e0b; font-weight: bold; }}
.chart-container {{ text-align: center; margin: 5mm 0; page-break-inside: avoid; }}
.chart-container img {{ max-width: 100%; height: auto; }}
.chart-caption {{ font-size: 8.5pt; color: #94a3b8; font-style: italic; margin-top: 2mm; text-align: center; }}
.chart-half {{ display: inline-block; width: 48%; vertical-align: top; text-align: center; margin: 2mm 0; }}
.chart-half img {{ max-width: 100%; height: auto; }}
.two-col {{ display: table; width: 100%; table-layout: fixed; margin: 3mm 0; }}
.two-col .col {{ display: table-cell; vertical-align: top; padding: 0 2mm; }}
.metric-card {{ background: #f8fafc; border: 1px solid #e2e8f0; border-radius: 6px; padding: 3mm 4mm; text-align: center; margin: 2mm 0; }}
.metric-card .value {{ font-size: 18pt; font-weight: bold; line-height: 1.2; }}
.metric-card .label {{ font-size: 8pt; color: #64748b; text-transform: uppercase; letter-spacing: 0.5px; }}
.action-item {{ background: #f8fafc; border-radius: 4px; padding: 3mm 4mm; margin: 3mm 0; border-left: 3px solid #cbd5e1; page-break-inside: avoid; }}
.action-item.critical {{ border-left-color: #ef4444; background: #fef2f2; }}
.action-item.high {{ border-left-color: #f59e0b; background: #fffbeb; }}
.action-item.medium {{ border-left-color: #1a56db; background: #eff6ff; }}
.priority-tag {{ display: inline-block; padding: 0.5mm 3mm; border-radius: 3px; font-size: 8pt; font-weight: bold; color: white; margin-right: 2mm; }}
.priority-critical {{ background: #ef4444; }}
.priority-high {{ background: #f59e0b; }}
.priority-medium {{ background: #1a56db; }}
.data-freshness {{ font-size: 8pt; color: #94a3b8; font-style: italic; margin-top: 4mm; padding-top: 2mm; border-top: 1px solid #e2e8f0; }}
"""
# ─── Section Builders ────────────────────────────────────────────────────────
def _img(path):
"""Convert file path to file:// URI for WeasyPrint."""
if not path:
return ""
return f'<div class="chart-container"><img src="file://{path}"></div>'
def _metric_card(value, label, color=None):
style = f' style="color: {color};"' if color else ""
return f'<div class="metric-card"><div class="value"{style}>{value}</div><div class="label">{label}</div></div>'
def _rating_class(rating):
r = str(rating).lower()
if "good" in r or "pass" in r:
return "status-pass"
elif "poor" in r or "fail" in r:
return "status-fail"
return "status-warn"
def section_title_page(domain, report_title, subtitle, score=None, meta_items=None):
score_html = ""
if score is not None:
score_html = f'''
<div class="score-box">
<div class="score-number">{score}</div>
<div class="score-label">Lighthouse Performance Score</div>
</div>'''
meta_html = ""
if meta_items:
spans = " • ".join(f"<span>{item}</span>" for item in meta_items)
meta_html = f'<div class="meta">{spans}</div>'
return f'''
<div class="title-page">
<div class="badge">{report_title}</div>
<h1>Google SEO Report</h1>
<div class="subtitle">{subtitle}</div>
<div class="url">{domain}</div>
{score_html}
{meta_html}
</div>'''
def section_cwv_audit(psi_data, crux_data, charts, history_data=None):
"""Build the Core Web Vitals audit section."""
html = '<div class="section"><div class="section-header"><h2>Core Web Vitals Audit</h2></div>'
# Lighthouse scores
psi = psi_data if isinstance(psi_data, dict) else {}
mobile = psi.get("psi", {}).get("mobile", psi)
scores = mobile.get("lighthouse_scores", {})
if scores:
html += '<h3>Lighthouse Scores</h3>'
html += charts.get("gauges", "")
# Lab metrics
lab = mobile.get("lab_metrics", {})
if lab:
html += '<h3>Lab Metrics</h3><table><thead><tr><th>Metric</th><th>Value</th><th>Score</th></tr></thead><tbody>'
for k, v in lab.items():
score_val = v.get("score")
score_pct = f"{score_val:.0%}" if score_val is not None else "N/A"
cls = "status-pass" if score_val and score_val >= 0.9 else ("status-warn" if score_val and score_val >= 0.5 else "status-fail")
html += f'<tr><td>{k}</td><td>{v.get("display", "")}</td><td class="{cls}">{score_pct}</td></tr>'
html += '</tbody></table>'
# CrUX field data
crux = crux_data if isinstance(crux_data, dict) else {}
crux_metrics = crux.get("metrics", {})
if crux_metrics:
html += '<h3>CrUX Field Data (28-day Rolling Average)</h3>'
html += charts.get("distributions", "")
html += '<table><thead><tr><th>Metric</th><th>p75</th><th>Rating</th><th>Good %</th><th>NI %</th><th>Poor %</th></tr></thead><tbody>'
for name, m in crux_metrics.items():
rating = m.get("rating", "?")
dist = m.get("distribution", {})
unit = m.get("unit", "")
p75 = m.get("p75", "?")
display_val = f"{p75:.3f}" if name == "cumulative_layout_shift" else f"{p75}{unit}"
html += f'<tr><td>{m.get("label", name)}</td><td>{display_val}</td>'
html += f'<td class="{_rating_class(rating)}">{rating.upper()}</td>'
html += f'<td>{dist.get("good", "N/A")}%</td><td>{dist.get("needs_improvement", "N/A")}%</td><td>{dist.get("poor", "N/A")}%</td></tr>'
html += '</tbody></table>'
cp = crux.get("collection_period", {})
if cp:
html += f'<p class="data-freshness">Collection period: {cp.get("first", "?")} to {cp.get("last", "?")}. CrUX data is a 28-day rolling average updated daily ~04:00 UTC.</p>'
elif crux.get("error"):
html += f'<div class="highlight"><strong>CrUX Field Data:</strong> {crux["error"]}</div>'
# CrUX History timeline
if history_data and not history_data.get("error"):
html += '<h3>Core Web Vitals Trends (25-week)</h3>'
html += charts.get("timeline", "")
trends = history_data.get("trends", {})
if trends:
html += '<table><thead><tr><th>Metric</th><th>Direction</th><th>Change</th><th>Earliest Avg</th><th>Latest Avg</th></tr></thead><tbody>'
for name, t in trends.items():
direction = t.get("direction", "?")
cls = "status-pass" if direction == "improving" else ("status-fail" if direction == "degrading" else "")
html += f'<tr><td>{t.get("label", name)}</td><td class="{cls}">{direction.upper()}</td>'
html += f'<td>{t.get("change_pct", 0):+.1f}%</td><td>{t.get("earliest_avg", "?")}</td><td>{t.get("latest_avg", "?")}</td></tr>'
html += '</tbody></table>'
# Failed audits
failed = mobile.get("failed_audits", [])
if failed:
html += f'<h3>Failed / Warning Audits ({len(failed)})</h3>'
html += '<table><thead><tr><th>Audit</th><th>Score</th><th>Details</th></tr></thead><tbody>'
for a in failed[:20]:
score_pct = f"{a['score']:.0%}" if a.get("score") is not None else "?"
html += f'<tr><td>{a.get("title", "")}</td><td class="status-fail">{score_pct}</td><td>{a.get("display", "")}</td></tr>'
html += '</tbody></table>'
# SEO audits
seo_audits = mobile.get("seo_audits", [])
if seo_audits:
seo_failed = [a for a in seo_audits if not a.get("pass")]
if seo_failed:
html += f'<h3>SEO Audit Issues ({len(seo_failed)})</h3>'
for a in seo_failed:
html += f'<div class="action-item critical"><h4>{a.get("title", "")}</h4></div>'
else:
html += f'<div class="success-box"><strong>SEO:</strong> All {len(seo_audits)} Lighthouse SEO checks passed.</div>'
# Accessibility issues
a11y = mobile.get("accessibility_audits", [])
if a11y:
html += f'<h3>Accessibility Issues ({len(a11y)})</h3>'
html += '<table><thead><tr><th>Issue</th><th>Score</th></tr></thead><tbody>'
for a in a11y:
html += f'<tr><td>{a.get("title", "")}</td><td class="status-fail">{a.get("score", 0):.0%}</td></tr>'
html += '</tbody></table>'
# Opportunities
opps = mobile.get("opportunities", [])
if opps:
html += f'<h3>Optimization Opportunities ({len(opps)})</h3>'
html += '<table><thead><tr><th>Opportunity</th><th>Estimated Savings</th></tr></thead><tbody>'
for o in opps:
html += f'<tr><td>{o.get("title", "")}</td><td>{o.get("savings_ms", 0)}ms</td></tr>'
html += '</tbody></table>'
html += '</div>'
return html
def section_gsc_performance(gsc_data, charts):
"""Build the GSC performance section."""
html = '<div class="section"><div class="section-header"><h2>Search Console Performance</h2></div>'
totals = gsc_data.get("totals", {})
dr = gsc_data.get("date_range", {})
if totals:
html += f'<p>Period: {dr.get("start", "?")} to {dr.get("end", "?")} | Property: {gsc_data.get("property", "?")}</p>'
clicks_val = f'{totals.get("clicks", 0):,}'
impr_val = f'{totals.get("impressions", 0):,}'
ctr_val = f'{totals.get("ctr", 0)}%'
rows_val = str(gsc_data.get("row_count", 0))
html += '<div class="two-col">'
html += f'<div class="col">{_metric_card(clicks_val, "Total Clicks", BRAND["primary"])}</div>'
html += f'<div class="col">{_metric_card(impr_val, "Total Impressions", BRAND["secondary"])}</div>'
html += '</div><div class="two-col">'
html += f'<div class="col">{_metric_card(ctr_val, "Average CTR", BRAND["accent"])}</div>'
html += f'<div class="col">{_metric_card(rows_val, "Queries Found")}</div>'
html += '</div>'
# Top queries chart
html += charts.get("top_queries", "")
# Top queries table
rows = gsc_data.get("rows", [])
if rows:
html += '<h3>Top Queries</h3>'
html += '<table><thead><tr><th>#</th><th>Query</th><th>Clicks</th><th>Impressions</th><th>CTR</th><th>Position</th></tr></thead><tbody>'
sorted_rows = sorted(rows, key=lambda r: r.get("clicks", 0), reverse=True)
for i, r in enumerate(sorted_rows[:25], 1):
query = r.get("query", r.get("keys", ["?"])[0])
html += f'<tr><td>{i}</td><td>{query}</td><td>{r.get("clicks", 0)}</td><td>{r.get("impressions", 0):,}</td>'
html += f'<td>{r.get("ctr", 0)}%</td><td>{r.get("position", 0)}</td></tr>'
html += '</tbody></table>'
# Quick wins
qw = gsc_data.get("quick_wins", [])
if qw:
html += f'<h3>Quick Wins ({len(qw)} opportunities)</h3>'
html += '<div class="highlight">These queries rank at position 4-10 with high impressions. A small ranking improvement could yield significant traffic gains.</div>'
html += '<table><thead><tr><th>Query</th><th>Position</th><th>Impressions</th><th>Clicks</th></tr></thead><tbody>'
for w in qw:
query = w.get("keys", ["?"])[0] if w.get("keys") else "?"
html += f'<tr><td>{query}</td><td>{w.get("position", 0)}</td><td>{w.get("impressions", 0):,}</td><td>{w.get("clicks", 0)}</td></tr>'
html += '</tbody></table>'
html += f'<p class="data-freshness">Search Analytics data has a 2-3 day lag. Data available for ~16 months.</p>'
html += '</div>'
return html
def section_indexation(inspect_data, charts):
"""Build the indexation status section."""
html = '<div class="section"><div class="section-header"><h2>Indexation Status</h2></div>'
summary = inspect_data.get("summary", {})
total = inspect_data.get("total", 0)
if summary:
html += charts.get("index_status", "")
html += f'<p>Total URLs inspected: {total}</p>'
html += '<div class="two-col">'
html += f'<div class="col">{_metric_card(summary.get("pass", 0), "Indexed", BRAND["success"])}</div>'
html += f'<div class="col">{_metric_card(summary.get("fail", 0), "Not Indexed", BRAND["danger"])}</div>'
html += '</div>'
results = inspect_data.get("results", [])
if results:
html += '<h3>Per-URL Results</h3>'
html += '<table><thead><tr><th>URL</th><th>Verdict</th><th>Coverage</th><th>Last Crawl</th></tr></thead><tbody>'
for r in results:
verdict = r.get("verdict", "?")
cls = "status-pass" if verdict == "PASS" else ("status-fail" if verdict == "FAIL" else "")
idx = r.get("index_status", {})
cov = idx.get("coverage_state", r.get("error", "N/A"))
crawl = idx.get("last_crawl_time", "N/A")
if crawl and crawl != "N/A":
crawl = crawl[:10]
html += f'<tr><td style="word-break:break-all;font-size:8pt;">{r.get("url", "?")}</td>'
html += f'<td class="{cls}">{verdict}</td><td>{cov}</td><td>{crawl}</td></tr>'
html += '</tbody></table>'
html += f'<p class="data-freshness">URL Inspection API: 2,000 inspections/day per site.</p>'
html += '</div>'
return html
# ─── Report Assemblers ───────────────────────────────────────────────────────
def generate_report(report_type, data, domain, output_dir, output_format="pdf"):
"""
Generate a complete report.
Args:
report_type: 'cwv-audit', 'gsc-performance', 'indexation', or 'full'.
data: Dictionary with all input data.
domain: Domain name for the report header.
output_dir: Directory for output files.
output_format: 'pdf', 'html', or 'both'.
Returns:
Dictionary with output paths.
"""
output_dir = Path(output_dir)
charts_dir = output_dir / "charts"
charts_dir.mkdir(parents=True, exist_ok=True)
timestamp = datetime.now().strftime("%Y-%m-%d %H:%M")
result = {"report_type": report_type, "domain": domain, "files": [], "error": None}
# Generate charts based on report type
chart_paths = {}
if report_type in ("cwv-audit", "full"):
psi = data.get("psi", data)
mobile = psi.get("psi", {}).get("mobile", psi) if isinstance(psi, dict) else {}
chart_paths["gauges"] = _img(chart_lighthouse_gauges(mobile, charts_dir))
crux = data.get("crux", {})
chart_paths["distributions"] = _img(chart_cwv_distributions({"crux": crux} if crux else data, charts_dir))
history = data.get("crux_history", {})
if history and not history.get("error"):
chart_paths["timeline"] = _img(chart_cwv_timeline(history, charts_dir))
if report_type in ("gsc-performance", "full"):
gsc = data.get("gsc", data)
chart_paths["top_queries"] = _img(chart_top_queries(gsc, charts_dir))
if report_type in ("indexation", "full"):
inspect = data.get("inspection", data)
chart_paths["index_status"] = _img(chart_index_status(inspect, charts_dir))
# Build HTML sections
sections = []
# Title page
if report_type == "cwv-audit":
mobile = data.get("psi", data).get("psi", {}).get("mobile", data) if isinstance(data, dict) else {}
perf_score = mobile.get("lighthouse_scores", {}).get("performance")
sections.append(section_title_page(domain, "Core Web Vitals Audit", "Performance & User Experience Analysis",
score=perf_score, meta_items=[timestamp, "PSI + CrUX"]))
sections.append(section_cwv_audit(data, data.get("crux", {}), chart_paths, data.get("crux_history")))
elif report_type == "gsc-performance":
gsc = data.get("gsc", data)
clicks = gsc.get("totals", {}).get("clicks", 0)
sections.append(section_title_page(domain, "Search Console Performance", "Google Search Analytics Report",
score=clicks, meta_items=[timestamp, "Google Search Console API"]))
sections.append(section_gsc_performance(gsc, chart_paths))
elif report_type == "indexation":
inspect = data.get("inspection", data)
total = inspect.get("total", 0)
sections.append(section_title_page(domain, "Indexation Status Report", "URL Index Coverage Analysis",
score=total, meta_items=[timestamp, "URL Inspection API"]))
sections.append(section_indexation(inspect, chart_paths))
elif report_type == "full":
sections.append(section_title_page(domain, "Google SEO Intelligence Report", "Comprehensive Analysis",
meta_items=[timestamp, "All Google APIs"]))
if data.get("psi") or data.get("crux"):
sections.append(section_cwv_audit(data.get("psi", {}), data.get("crux", {}), chart_paths, data.get("crux_history")))
if data.get("gsc"):
sections.append(section_gsc_performance(data["gsc"], chart_paths))
if data.get("inspection"):
sections.append(section_indexation(data["inspection"], chart_paths))
# Assemble HTML
css = _base_css(domain)
body = "\n".join(sections)
html_content = f"""<!DOCTYPE html><html lang="en"><head><meta charset="UTF-8"><style>{css}</style></head><body>{body}</body></html>"""
# Output (closes audit VULN-014 path traversal on --domain)
# Whitelist: alphanumerics + dot + underscore + hyphen. Anything else
# (incl. "..", "/", "\\", NUL, Windows drive letters) becomes "_".
import re
safe_domain = re.sub(r"[^A-Za-z0-9._-]", "_", domain)[:128] or "report"
base_name = f"Google-SEO-Report-{safe_domain}-{report_type}"
# Re-resolve and assert containment inside output_dir (defense in depth).
out_root = Path(output_dir).resolve()
def _safe_path(name: str) -> Path:
candidate = (out_root / name).resolve()
if out_root != candidate.parent and out_root not in candidate.parents:
raise ValueError(
f"Refusing to write outside output_dir: {candidate}"
)
return candidate
if output_format in ("html", "both"):
html_path = _safe_path(f"{base_name}.html")
with open(html_path, "w") as f:
f.write(html_content)
result["files"].append(str(html_path))
if output_format in ("pdf", "both"):
pdf_path = _safe_path(f"{base_name}.pdf")
try:
HTML(string=html_content).write_pdf(str(pdf_path))
result["files"].append(str(pdf_path))
except Exception as e:
result["error"] = f"PDF generation failed: {e}"
return result
# ─── CLI ─────────────────────────────────────────────────────────────────────
def main():
parser = argparse.ArgumentParser(
description="Google SEO Report Generator - Professional PDF/HTML reports"
)
parser.add_argument(
"--type", "-t",
choices=["cwv-audit", "gsc-performance", "indexation", "full"],
required=True,
help="Report type",
)
parser.add_argument("--data", "-d", help="Path to JSON data file (or pipe via stdin)")
parser.add_argument("--domain", required=True, help="Domain name for the report header")
parser.add_argument("--output-dir", "-o", default=".", help="Output directory (default: current)")
parser.add_argument(
"--format", "-f",
choices=["pdf", "html", "both"],
default="pdf",
help="Output format (default: pdf)",
)
parser.add_argument("--json", "-j", action="store_true", help="Output metadata as JSON")
args = parser.parse_args()
# Load data
if args.data:
try:
with open(args.data, "r") as f:
data = json.load(f)
except (json.JSONDecodeError, IOError) as e:
print(f"Error reading data file: {e}", file=sys.stderr)
sys.exit(1)
elif not sys.stdin.isatty():
try:
data = json.load(sys.stdin)
except json.JSONDecodeError as e:
print(f"Error parsing stdin JSON: {e}", file=sys.stderr)
sys.exit(1)
else:
print("Error: Provide --data file or pipe JSON via stdin.", file=sys.stderr)
sys.exit(1)
result = generate_report(
report_type=args.type,
data=data,
domain=args.domain,
output_dir=args.output_dir,
output_format=args.format,
)
if result.get("error"):
print(f"Error: {result['error']}", file=sys.stderr)
if args.json:
print(json.dumps(result, indent=2))
else:
for f in result.get("files", []):
print(f"Generated: {f}")
if __name__ == "__main__":
main()
# Blog Google Skill Dependencies
# Installed in the skill's local .venv via run.py
#
# For reproducible installs with hash verification, use the lock file:
# pip install --require-hashes -r requirements.lock
# This requirements.txt declares acceptable version ranges; the lock file
# pins exact versions + sha256 hashes for every transitive dep (closes
# audit VULN-006 supply-chain detection gap).
# Core Google API client
google-api-python-client>=2.100.0,<3.0.0
google-auth>=2.20.0,<3.0.0
google-auth-oauthlib>=1.0.0,<2.0.0
google-auth-httplib2>=0.2.0,<1.0.0
# GA4 Data API
google-analytics-data>=0.18.0,<1.0.0
# HTTP requests (PSI, CrUX, NLP)
requests>=2.32.4,<3.0.0
# Report generation (optional - graceful fallback if unavailable)
matplotlib>=3.8.0,<4.0.0
weasyprint>=61.0,<70.0
#!/usr/bin/env python3
"""
Universal runner for Blog Google skill scripts
Ensures all scripts run with the correct virtual environment
"""
import os
import sys
import subprocess
from pathlib import Path
def get_venv_python():
"""Get the virtual environment Python executable"""
skill_dir = Path(__file__).parent.parent
venv_dir = skill_dir / ".venv"
if os.name == 'nt': # Windows
venv_python = venv_dir / "Scripts" / "python.exe"
else: # Unix/Linux/Mac
venv_python = venv_dir / "bin" / "python"
return venv_python
def ensure_venv():
"""Ensure virtual environment exists"""
skill_dir = Path(__file__).parent.parent
venv_dir = skill_dir / ".venv"
setup_script = skill_dir / "scripts" / "setup_environment.py"
# Check if venv exists
if not venv_dir.exists():
print("First-time setup: Creating virtual environment...")
print(" This may take a minute...")
# Run setup with system Python
result = subprocess.run([sys.executable, str(setup_script)])
if result.returncode != 0:
print("Failed to set up environment")
sys.exit(1)
print("Environment ready!")
return get_venv_python()
def main():
"""Main runner"""
if len(sys.argv) < 2:
print("Usage: python run.py <script_name> [args...]")
print("\nAvailable scripts:")
print(" google_auth.py - Credential management and auth setup")
print(" pagespeed_check.py - PageSpeed Insights + CrUX field data")
print(" crux_history.py - 25-week CWV trend history")
print(" youtube_search.py - YouTube video search and details")
print(" nlp_analyze.py - NLP entity extraction and sentiment")
print(" gsc_query.py - Search Console performance data")
print(" gsc_inspect.py - URL Inspection API")
print(" indexing_notify.py - Indexing API notifications")
print(" ga4_report.py - GA4 organic traffic reports")
print(" keyword_planner.py - Google Ads Keyword Planner")
print(" google_report.py - PDF/HTML performance reports")
sys.exit(1)
script_name = sys.argv[1]
script_args = sys.argv[2:]
# Handle both "scripts/script.py" and "script.py" formats
if script_name.startswith('scripts/'):
script_name = script_name[8:] # len('scripts/') = 8
# Ensure .py extension
if not script_name.endswith('.py'):
script_name += '.py'
# Get script path
skill_dir = Path(__file__).parent.parent
script_path = skill_dir / "scripts" / script_name
if not script_path.exists():
print(f"Script not found: {script_name}")
print(f" Skill directory: {skill_dir}")
print(f" Looked for: {script_path}")
sys.exit(1)
# Ensure venv exists and get Python executable
venv_python = ensure_venv()
# Build command
cmd = [str(venv_python), str(script_path)] + script_args
# Run the script
try:
result = subprocess.run(cmd)
sys.exit(result.returncode)
except KeyboardInterrupt:
print("\nInterrupted by user")
sys.exit(130)
except Exception as e:
print(f"Error: {e}")
sys.exit(1)
if __name__ == "__main__":
main()