
Ecommerce Competitor Analyzer
- 1.4k installs
- 51 repo stars
- Updated February 3, 2026
- buluslan/ecommerce-competitor-analyzer
ecommerce-competitor-analyzer is an agent skill that multi-platform e-commerce competitor analysis skill that automatically scrapes product data from amazon, temu, shopee and generates comprehensive analysis reports usin
About
ecommerce-competitor-analyzer is an agent skill from buluslan/ecommerce-competitor-analyzer that multi-platform e-commerce competitor analysis skill that automatically scrapes product data from amazon, temu, shopee and generates comprehensive analysis reports using ai. use when you need to analyz. # E-commerce Competitor Analyzer Skill ## Quick Start (For AI) **When to use this skill**: When user asks to analyze, research, or extract insights from e-commerce products (Amazon, Temu, Shopee). **What you should do**: 1. Extract product identifiers (ASINs or URLs) from user input 2. Call the scraper script to get product data 3. Call the AI a Developers invoke ecommerce-competitor-analyzer during build/integrations work for ai & agent building tasks. The skill documents triggers, prerequisites, and step-by-step workflows grounded in SKILL.md. Compatible with Claude Code, Cursor, and Codex agent runtimes that load marketplace skills.
- E-commerce Competitor Analyzer Skill
- When to use this skill**: When user asks to analyze, research, or extract insights from e-commerce products (Amazon, Tem
- 1. Extract product identifiers (ASINs or URLs) from user input
- 2. Call the scraper script to get product data
- 3. Call the AI analysis with the analysis prompt template
Ecommerce Competitor Analyzer by the numbers
- 1,350 all-time installs (skills.sh)
- +8 installs in the week ending Aug 5, 2026 (Skillselion tracking)
- Ranked #871 of 16,546 AI & Agent Building skills by installs in the Skillselion catalog
- Security screen: MEDIUM risk (skills.sh audit)
- Data as of Aug 5, 2026 (Skillselion catalog sync)
ecommerce-competitor-analyzer capabilities & compatibility
- Capabilities
- e commerce competitor analyzer skill · when to use this skill**: when user asks to anal · 1. extract product identifiers (asins or urls) f · 2. call the scraper script to get product data · 3. call the ai analysis with the analysis prompt
- Use cases
- orchestration
What ecommerce-competitor-analyzer says it does
**When to use this skill**: When user asks to analyze, research, or extract insights from e-commerce products (Amazon, Temu, Shopee).
1. Extract product identifiers (ASINs or URLs) from user input
2. Call the scraper script to get product data
npx skills add https://github.com/buluslan/ecommerce-competitor-analyzer --skill ecommerce-competitor-analyzerAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 1.4k |
|---|---|
| repo stars | ★ 51 |
| Security audit | 2 / 3 scanners passed |
| Last updated | February 3, 2026 |
| Repository | buluslan/ecommerce-competitor-analyzer ↗ |
What it does
Multi-platform e-commerce competitor analysis skill that automatically scrapes product data from Amazon, Temu, Shopee and generates comprehensive analysis reports using AI. Use when you need to analyz
Who is it for?
Developers working on ai & agent building during build tasks.
Skip if: Tasks outside AI & Agent Building scope described in SKILL.md.
When should I use this skill?
Multi-platform e-commerce competitor analysis skill that automatically scrapes product data from Amazon, Temu, Shopee and generates comprehensive analysis reports using AI. Use when you need to analyz
What you get
Completed ai & agent building workflow aligned with SKILL.md steps.
- competitive analysis report
- structured listing dataset
By the numbers
- Supports 3 ecommerce platforms: Amazon, Temu, and Shopee
- Skill version 1.0.0 in buluslan/ecommerce-competitor-analyzer
Files
E-commerce Competitor Analyzer Skill
Quick Start (For AI)
When to use this skill: When user asks to analyze, research, or extract insights from e-commerce products (Amazon, Temu, Shopee).
What you should do: 1. Extract product identifiers (ASINs or URLs) from user input 2. Call the scraper script to get product data 3. Call the AI analysis with the analysis prompt template 4. Output results in BOTH formats: Google Sheets + Markdown
Input examples:
- "Analyze B0C4YT8S6H"
- "Analyze these products: B0C4YT8S6H, B08N5WRQ1Y, B0CLFH7CCV"
- "Research this competitor: https://amazon.com/dp/B0C4YT8S6H"
Output requirements:
- Google Sheets table with: ASIN, Title, Price, Rating, 4 analysis summaries
- Markdown report with detailed 4-dimensional analysis
---
How AI Should Process Requests
Step 1: Extract Product Identifiers
From user input, extract all ASINs and/or URLs:
Example inputs:
"Analyze these Amazon products:
B0C4YT8S6H
B08N5WRQ1Y
B0CLFH7CCV"Extract: ['B0C4YT8S6H', 'B08N5WRQ1Y', 'B0CLFH7CCV']
Mixed input handling:
"Analyze B0C4YT8S6H and https://amazon.com/dp/B08N5WRQ1Y"Extract: ['B0C4YT8S6H', 'B08N5WRQ1Y'] (extract ASIN from URL)
Step 2: Batch Scrape Product Data
For each product identifier: 1. Detect platform (use scripts/detect-platform.js if available) 2. Call appropriate scraper (Amazon: scripts/scrape-amazon.js) 3. Use Olostep API with configured API key from .env
Batch processing pattern:
// Process all products in parallel
const products = ['B0C4YT8S6H', 'B08N5WRQ1Y', 'B0CLFH7CCV'];
const results = await Promise.allSettled(
products.map(asin => scrapeAmazon(asin))
);
// Handle failures gracefully
const successful = results.filter(r => r.status === 'fulfilled');
const failed = results.filter(r => r.status === 'rejected');Step 3: Batch AI Analysis
For each successfully scraped product: 1. Read the analysis prompt from prompts/analysis-prompt-base.md 2. Replace product data placeholders in the prompt 3. Call Gemini API (model: gemini-3-flash-preview) 4. Extract structured analysis results
Analysis framework (4 dimensions): 1. 文案构建逻辑与词频分析 (The Brain) - Copywriting strategy & keywords 2. 视觉资产设计思路 (The Face) - Visual design methodology 3. 评论定量与定性分析 (The Voice) - Review sentiment analysis 4. 市场维态与盲区扫描 (The Pulse) - Market positioning & blind spots
Step 4: Generate Dual Format Output
Format 1: Google Sheets (Structured Data)
Write to Google Sheets with columns: | ASIN | 产品标题 | 价格 | 评分 | 文案分析摘要 | 视觉分析摘要 | 评论分析摘要 | 市场分析摘要 |
Sheet selection priority: 1. User explicitly specified Sheet ID/Name/URL 2. Default from .env (GOOGLE_SHEETS_ID) 3. Ask user to provide Sheet ID
Format 2: Markdown Report (Detailed Analysis)
Generate file: 竞品分析-YYYY-MM-DD.md
Structure:
# Amazon Competitor Analysis Report
## Analysis Overview
- Products analyzed: 3
- Analysis date: 2026-01-29
- Total time: ~5 minutes
---
## Product 1: B0C4YT8S6H
### Basic Information
- Title: [Product title]
- Price: [Price]
- Rating: [Rating]
### Copywriting Strategy & Keyword Analysis
[Full analysis...]
### Visual Asset Design Methodology
[Full analysis...]
### Customer Review Analysis
[Full analysis...]
### Market Positioning & Competitive Intelligence
[Full analysis...]
------
File Structure
ecommerce-competitor-analyzer.skill/
├── SKILL.md # This file (AI instructions)
├── platforms.yaml # Platform configurations (URL patterns, regex)
├── .env.example # Configuration template (API keys)
├── prompts/ # AI prompt templates
│ └── analysis-prompt-base.md # Base analysis framework (from n8n)
├── scripts/ # Processing scripts
│ ├── detect-platform.js # Platform detection utility
│ ├── scrape-amazon.js # Amazon scraper (Olostep API)
│ └── batch-processor.js # Batch processing engine
└── references/ # Documentation
└── n8n-workflow-analysis.md # n8n workflow insights---
Configuration Files
platforms.yaml
Contains platform-specific configurations:
- URL patterns for platform detection
- ASIN extraction regex patterns
- Scraper API endpoints
- Data extraction patterns
Key sections:
platforms:
amazon:
url_patterns: ["amazon.com", "amazon.co.uk", ...]
asin_regex:
standard: "/dp/([A-Z0-9]{10})"
scraper:
provider: "olostep"
api_endpoint: "https://api.olostep.com/v2/agent/web-agent".env.example
Template for required API keys:
OLOSTEP_API_KEY=your_olostep_api_key_here
GEMINI_API_KEY=your_gemini_api_key_here
GOOGLE_SHEETS_ID=YOUR_GOOGLE_SHEETS_ID_HERECritical: Always check if .env file exists and contains required keys before processing.
---
Analysis Prompt Template
The AI analysis uses a proven 4-dimensional framework. The exact prompt is stored in: prompts/analysis-prompt-base.md
Key sections: 1. Role: 10-year experienced Amazon Operations Director & Brand Strategist 2. Goal: Deep scan of product listing to extract strategic insights 3. Output Structure:
- Part 1: 文案构建逻辑与词频分析
- Part 2: 视觉资产设计思路
- Part 3: 评论定量与定性分析
- Part 4: 市场维态与盲区扫描
Important: Use the prompt EXACTLY as provided in the template without modifications.
---
API Services
Olostep API (Web Scraping)
- Purpose: Scrape Amazon product pages with rendered JavaScript
- Endpoint:
https://api.olostep.com/v2/agent/web-agent - Cost: 1000 free requests/month, then $0.002/request
- Key param:
comments_to_scrape: 100(matching n8n config)
Google Gemini API (AI Analysis)
- Purpose: Generate comprehensive product analysis
- Model:
gemini-3-flash-preview(cost-effective) - Cost: ~$0.001/product
- Alternative:
gemini-2-flash-thinking(for complex analysis)
Google Sheets API (Data Storage)
- Purpose: Export structured results
- Authentication: OAuth2 service account
- Cost: Free tier
---
Error Handling
Batch Processing with Error Isolation
Critical pattern from n8n workflow:
const items = productIdentifiers;
const results = await Promise.allSettled(
items.map(async (item, index) => {
try {
const data = await scrapeProduct(item);
const analysis = await analyzeWithAI(data);
return { success: true, index, data: analysis };
} catch (error) {
// Single failure doesn't stop batch
return { success: false, index, error: error.message };
}
})
);
// Report results
const successful = results.filter(r => r.status === 'fulfilled' && r.value.success);
const failed = results.filter(r => r.status === 'rejected' || !r.value.success);
console.log(`Processed: ${successful.length} succeeded, ${failed.length} failed`);Common Errors & Solutions
| Error | Cause | Solution |
|---|---|---|
OLOSTEP_API_KEY not found | Missing .env file | Check .env exists and contains key |
Invalid ASIN format | Malformed ASIN | Validate ASIN: 10 alphanumeric chars |
Scraping timeout | Slow page load | Increase timeout or retry |
Gemini rate limit | Too many requests | Add delay between batches |
---
Platform Detection Logic
function detectPlatform(urlOrId) {
// Direct ASIN
if (/^[A-Z0-9]{10}$/.test(urlOrId)) {
return { platform: 'amazon', id: urlOrId };
}
// Amazon URL patterns
if (/amazon\.(com|co\.uk|de|es|fr|it|ca|co\.jp)/i.test(urlOrId)) {
const asinMatch = urlOrId.match(/\/dp\/([A-Z0-9]{10})/i);
if (asinMatch) {
return { platform: 'amazon', id: asinMatch[1] };
}
}
// Other platforms (future)
// if (/temu\.com/i.test(urlOrId)) return { platform: 'temu', id: extractId(urlOrId) };
return null;
}---
Implementation Notes
Current Version: Phase 1 MVP
Supported Platforms: Amazon (US only) Input Method: Dialog-based (ASINs or URLs) Output Format: Google Sheets table + Markdown report
Roadmap
- ✅ Phase 1: Amazon MVP (current)
- 🔄 Phase 2: Add Temu & Shopee platforms
- 🔄 Phase 3: Cross-platform comparison
- 🔄 Phase 4: Historical tracking & price alerts
Design Philosophy
This skill follows the error isolation pattern from the n8n workflow:
- Single product failure NEVER stops the entire batch
- Always report both successes and failures
- Provide detailed error messages for debugging
Performance Benchmarks
| Operation | Time | Cost |
|---|---|---|
| Single product scrape | ~30 seconds | $0.002 (Olostep) |
| Single product analysis | ~45 seconds | $0.001 (Gemini) |
| Total per product | ~1-2 minutes | ~$0.003 |
| Batch of 10 products | ~10-15 minutes (parallel) | ~$0.03 |
---
References
- n8n Workflow: Based on v81 workflow logic
- Platform Config: See
platforms.yamlfor URL patterns and extraction rules - Analysis Prompt: See
prompts/analysis-prompt-base.mdfor exact prompt template
---
Important Reminders for AI
1. ALWAYS extract ALL product identifiers from user input before processing 2. ALWAYS use batch processing with Promise.allSettled for error isolation 3. ALWAYS generate BOTH output formats: Google Sheets + Markdown 4. NEVER modify the analysis prompt - use it exactly as provided 5. ALWAYS validate .env exists before starting processing 6. ALWAYS report processing summary: X succeeded, Y failed 7. If Google Sheets ID is missing, ask user to provide it 8. Use the exact prompt from prompts/analysis-prompt-base.md without any modifications
# E-commerce Competitor Analyzer Skill - Environment Variables
# Copy this file to `.env` and fill in your actual values
# ===================================================================
# REQUIRED API KEYS
# ===================================================================
# Olostep API Key
# Get your API key from: https://olostep.com/
# Pricing: 1000 free requests/month, $0.002/request
# IMPORTANT: Include the "olostep_" prefix in your key
# Example: olostep_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
OLOSTEP_API_KEY=your_olostep_api_key_here
# Olostep API Version (v1 or v2)
# v1: /v1/scrapes - Recommended (do NOT use extract parameter)
# v2: /v2/agent/web-agent - Alternative if your key supports it
# Default: v1
# Recommendation: Use v1 without extract parameter for best accuracy
OLOSTEP_API_VERSION=v1
# Google Gemini API Key
# Get your API key from: https://aistudio.google.com/app/apikey
# Model used: gemini-3-flash-preview
GEMINI_API_KEY=your_gemini_api_key_here
# ===================================================================
# OPTIONAL CONFIGURATIONS
# ===================================================================
# Google Sheets API Credentials (OAuth2 JSON)
# Only needed if writing to Google Sheets
# See: https://console.cloud.google.com/apis/credentials
GOOGLE_SHEETS_CREDENTIALS='{"type":"service_account",...}'
# Default Google Sheets ID for output
GOOGLE_SHEETS_ID=YOUR_GOOGLE_SHEETS_ID_HERE
# Output directory for markdown reports
MARKDOWN_OUTPUT_DIR=./output
# ===================================================================
# ADVANCED SETTINGS
# ===================================================================
# Scraping settings
SCRAPER_COMMENTS_NUMBER=100
SCRAPER_TIMEOUT=120000
# Batch processing settings
MAX_BATCH_SIZE=20
CONCURRENCY_LIMIT=5
# ===================================================================
# SETUP INSTRUCTIONS
# ===================================================================
# 1. Olostep API Key:
# - Sign up at https://olostep.com/
# - Get API key from dashboard
# - Free tier: 1000 requests/month
#
# 2. Google Gemini API Key:
# - Visit https://aistudio.google.com/app/apikey
# - Click "Create API Key"
# - Copy the key
#
# 3. Google Sheets (Optional):
# - Create Google Cloud Project
# - Enable Google Sheets API
# - Create OAuth2 credentials
# - Download JSON credentials
# - Paste content (single line) above
# Environment variables (contains API keys)
.env
.env.local
.env.*.local
# OAuth2 tokens
.google-tokens.json
*.tokens.json
service-account-key.json
.google-credentials.json
# Output files (generated results)
output/*.json
reports/*.md
# System files
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db
# IDE
.vscode/
.idea/
*.swp
*.swo
*~
.project
.classpath
.settings/
# Logs
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*
# Node modules
node_modules/
# Test coverage
coverage/
.npm
# Temporary files
tmp/
temp/
*.tmp
# Development test scripts with hardcoded API keys
test-olostep-key.js
快速部署指南
🚀 一键安装命令
方法 1:Git 克隆(推荐)
# 克隆仓库到本地
git clone https://github.com/buluslan/ecommerce-competitor-analyzer.git
# 复制到 Claude Code skills 目录
cp -r ecommerce-competitor-analyzer ~/.claude/skills/main-mode-skills/ecommerce-competitor-analyzer.skill---
方法 2:软连接(推荐给开发者)
# 克隆仓库
git clone https://github.com/buluslan/ecommerce-competitor-analyzer.git
# 创建软连接(便于后续更新)
ln -s $(pwd)/ecommerce-competitor-analyzer ~/.claude/skills/main-mode-skills/ecommerce-competitor-analyzer.skill---
方法 3:直接下载(不需要 Git)
# 下载并解压
curl -L https://github.com/buluslan/ecommerce-competitor-analyzer/archive/refs/heads/main.zip -o ecommerce-competitor-analyzer.zip
unzip ecommerce-competitor-analyzer.zip
# 复制到 Claude Code skills 目录
cp -r ecommerce-competitor-analyzer-main ~/.claude/skills/main-mode-skills/ecommerce-competitor-analyzer.skill
# 清理
rm ecommerce-competitor-analyzer.zip---
⚙️ 安装后配置
1. 配置环境变量
# 进入 skill 目录
cd ~/.claude/skills/main-mode-skills/ecommerce-competitor-analyzer.skill
# 复制环境变量模板
cp .env.example .env
# 编辑 .env 文件,添加你的 API 密钥
nano .env
# 或使用 VSCode: code .env
# 或使用 Vim: vim .env2. 获取 API 密钥
必需的 API 密钥:
| 服务 | 获取地址 | 费用 |
|---|---|---|
| Olostep API | https://olostep.com/ | 1000次/月免费 |
| Google Gemini | https://aistudio.google.com/app/apikey | ~$0.001/产品 |
3. 验证安装
# 验证环境变量配置
cd ~/.claude/skills/main-mode-skills/ecommerce-competitor-analyzer.skill
node scripts/verify-env.js预期输出:
✅ OLOSTEP_API_KEY: Configured
✅ GEMINI_API_KEY: Configured
✅ GOOGLE_SHEETS_ID: Optional (not configured)
Environment setup complete!---
🎮 开始使用
配置完成后,在 Claude Code 中直接说:
分析这个 Amazon 产品:B0C4YT8S6H或批量分析:
分析这些 Amazon 产品:
B0C4YT8S6H
B08N5WRQ1Y
B0CLFH7CCV---
🔄 更新项目
如果你使用了软连接方式安装:
# 进入项目目录
cd /path/to/ecommerce-competitor-analyzer
# 拉取最新代码
git pull origin main如果你使用了复制方式安装:
# 删除旧版本
rm -rf ~/.claude/skills/main-mode-skills/ecommerce-competitor-analyzer.skill
# 重新克隆
git clone https://github.com/buluslan/ecommerce-competitor-analyzer.git
cp -r ecommerce-competitor-analyzer ~/.claude/skills/main-mode-skills/ecommerce-competitor-analyzer.skill---
🗑️ 卸载
# 删除 skill
rm -rf ~/.claude/skills/main-mode-skills/ecommerce-competitor-analyzer.skill
# 如果是软连接,删除链接
rm ~/.claude/skills/main-mode-skills/ecommerce-competitor-analyzer.skill---
📝 完整安装脚本(一键执行)
保存为 install.sh,然后运行 bash install.sh:
#!/bin/bash
echo "🚀 开始安装 E-commerce Competitor Analyzer Skill..."
# 克隆仓库
echo "📦 克隆仓库..."
git clone https://github.com/buluslan/ecommerce-competitor-analyzer.git
# 复制到 Claude Code skills 目录
echo "📋 安装到 Claude Code..."
mkdir -p ~/.claude/skills/main-mode-skills
cp -r ecommerce-competitor-analyzer ~/.claude/skills/main-mode-skills/ecommerce-competitor-analyzer.skill
# 配置环境变量
echo "⚙️ 配置环境变量..."
cd ~/.claude/skills/main-mode-skills/ecommerce-competitor-analyzer.skill
cp .env.example .env
echo ""
echo "✅ 安装完成!"
echo ""
echo "📝 下一步:"
echo "1. 编辑 ~/.claude/skills/main-mode-skills/ecommerce-competitor-analyzer.skill/.env"
echo "2. 添加你的 OLOSTEP_API_KEY 和 GEMINI_API_KEY"
echo "3. 运行: cd ~/.claude/skills/main-mode-skills/ecommerce-competitor-analyzer.skill && node scripts/verify-env.js"
echo ""
echo "📚 详细文档: https://github.com/buluslan/ecommerce-competitor-analyzer"---
❓ 常见问题
Q: Claude Code 找不到 skill?
A: 确保文件在正确的目录:~/.claude/skills/main-mode-skills/
Q: 如何查看已安装的 skills?
A:
ls ~/.claude/skills/main-mode-skills/Q: API 密钥在哪里配置?
A: 在 skill 目录下的 .env 文件中
---
📚 更多文档
环境变量配置指南
快速开始
# 1. 复制示例配置文件
cp .env.example .env
# 2. 编辑 .env 文件,填入你的 API Keys
nano .env # 或使用你喜欢的编辑器获取 API Keys
1. Olostep API Key(必需)
用途: 网页抓取服务(100条评论深度抓取)
获取步骤: 1. 访问: https://olostep.com/ 2. 注册账号 3. 登录后进入 Dashboard 4. 找到 API Key 部分 5. 复制你的 API Key
定价:
- 免费额度: 1000 次/月
- 超出后: $0.002/请求
配置:
OLOSTEP_API_KEY=olostep_xxxxxxxxxxxxx---
2. Google Gemini API Key(必需)
用途: AI 竞品分析(gemini-3-flash-preview)
获取步骤: 1. 访问: https://aistudio.google.com/app/apikey 2. 点击 "Create API Key" 或 "创建 API 密钥" 3. 选择或创建一个 Google Cloud 项目 4. 复制生成的 API Key
定价:
- 免费额度: 每月一定请求数
- 按量付费: 性价比高
配置:
GEMINI_API_KEY=AIzaSyXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX---
3. Google Sheets Credentials(可选)
用途: 自动写入分析结果到 Google Sheets
注意: 阶段 1 MVP 可以先不配置,使用 Markdown 输出
获取步骤: 1. 访问: https://console.cloud.google.com/apis/credentials 2. 选择一个项目或创建新项目 3. 点击 "Create Credentials" → "OAuth Client ID" 4. 应用类型选择 "Web application" 5. 添加授权重定向 URI:
http://localhost:80806. 下载 JSON 凭证文件 7. 将 JSON 内容(单行)复制到 .env
配置:
GOOGLE_SHEETS_CREDENTIALS='{"web":{"client_id":"...","client_secret":"...","auth_uri":"...","token_uri":"...","redirect_uris":["..."]}}'---
配置验证
配置完成后,运行验证脚本:
node scripts/verify-env.js预期输出:
✅ OLOSTEP_API_KEY: Configured
✅ GEMINI_API_KEY: Configured
✅ GOOGLE_SHEETS_CREDENTIALS: Optional (not configured)
Environment setup complete!---
测试 API Keys
测试 Olostep API
curl -X POST https://api.olostep.com/v2/agent/web-agent \
-H "Authorization: Bearer YOUR_OLOSTEP_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://www.amazon.com/dp/B0C4YT8S6H",
"comments_number": 10
}'测试 Gemini API
curl -X POST \
"https://generativelanguage.googleapis.com/v1beta/models/gemini-3-flash-preview:generateContent?key=YOUR_GEMINI_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"contents": [{
"parts": [{"text": "Hello"}]
}]
}'---
安全提醒
1. 永远不要将 .env 文件提交到 Git 2. 永远不要在公开代码中暴露 API Keys 3. 定期轮换你的 API Keys 4. 为不同环境使用不同的 Keys
---
故障排除
问题 1: Olostep API 返回 401
原因: API Key 无效或过期 解决: 检查 .env 中的 OLOSTEP_API_KEY 是否正确
问题 2: Gemini API 返回 403
原因: API Key 没有权限或配额用尽 解决: 检查 Google Cloud Console 中的配额和权限
问题 3: Google Sheets 认证失败
原因: OAuth2 凭证格式错误或重定向 URI 不匹配 解决: 确保重定向 URI 完全匹配(注意末尾不要有斜杠)
---
n8n 工作流凭证参考
如果你已经在 n8n 中配置过这些凭证,可以在 n8n 界面查看:
- n8n URL: Your n8n workflow URL (if applicable)
- Settings → Credentials
但是注意:n8n 使用加密的凭证存储,无法直接复制 API Keys。 你需要按照上述步骤重新获取。
---
下一步
环境变量配置完成后,你就可以使用 skill 了:
"分析 B0C4YT8S6H"或者批量分析:
"分析 B0C4YT8S6H, B08N5WRQ1Y, B0CLFH7CCV"MIT License
Copyright (c) 2026 Buluslan@新西楼Newest AI
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
# 电商平台配置
# 参考:n8n 工作流配置 (v81)
# 支持的平台
platforms:
amazon:
name: "Amazon"
version: "1.0.0"
status: "active"
enabled: true
# URL 匹配模式
url_patterns:
- "amazon.com"
- "amazon.co.uk"
- "amazon.de"
- "amazon.es"
- "amazon.fr"
- "amazon.it"
- "amazon.ca"
- "amazon.co.jp"
# ASIN 提取正则表达式
asin_regex:
standard: "/dp/([A-Z0-9]{10})"
product_page: "/([A-Z0-9]{10})/ref="
direct: "^[A-Z0-9]{10}$"
# 爬虫配置
scraper:
provider: "olostep"
api_endpoint: "https://api.olostep.com/v2/agent/web-agent"
comments_to_scrape: 100 # 匹配 n8n 配置
wait_time: 10
timeout: 120000
# 数据提取模式(来自 n8n "Code - 提取结构化数据" 节点)
extraction_patterns:
title:
- regex: "产品标题[::]+([^\\n]+)"
- regex: "Title[::]+([^\\n]+)"
- default: "未知"
price:
- regex: "价格[::]+[^0-9]*([0-9]+\\.?[0-9]*)"
- regex: "Price[::]+[^0-9]*([0-9]+\\.?[0-9]*)"
- default: "未知"
rating:
- regex: "评分[::]+[^0-9]*([0-9]+\\.?[0-9]*)"
- regex: "Rating[::]+[^0-9]*([0-9]+\\.?[0-9]*)"
- default: "未知"
# 输出格式(匹配 Google Sheets 结构)
output_format:
- "ASIN"
- "产品标题"
- "价格"
- "评分"
- "文案分析摘要"
- "视觉分析摘要"
- "评论分析摘要"
- "市场分析摘要"
# 未来平台(尚未实现)
temu:
name: "Temu"
version: "0.0.1"
status: "planned"
enabled: false
notes: "需要基于 Playwright 的爬虫"
shopee:
name: "Shopee"
version: "0.0.1"
status: "planned"
enabled: false
notes: "需要基于 Playwright 的爬虫"
# 全局设置
settings:
# 批处理限制
max_batch_size: 20
concurrency_limit: 5
retry_attempts: 3
# API 超时设置
scraping_timeout: 120000 # 2分钟
analysis_timeout: 60000 # 1分钟
# 错误处理
continue_on_error: true
require_all_success: false
# 输出设置
google_sheets_id: "YOUR_GOOGLE_SHEETS_ID_HERE"
markdown_output_dir: "./output"
Amazon Product Analysis Prompt
Source: n8n Workflow "Google Gemini" Node (v81)
File: 工作流配置.json (Lines 185-195)Status: Used exactly without modifications
---
Prompt Template
你是亚马逊竞品分析专家。请分析以下产品页面的内容:
{{ PRODUCT_CONTENT }}
Role / 身份角色
你是一位拥有 10 年经验的"亚马逊顶级运营总监"和"品牌战略官"。你不仅精通 A9和rufus算法,更擅长解析品牌背后的营销心理学与视觉工业设计逻辑。你的任务是透过 Listing 表面现象,还原对手的战略布局、设计方法论以及运营套路。
Goal / 工作目标
对 [提供的ASIN/商品链接/文本/图片] 进行深度扫描,输出一份能够指导产品迭代和营销升级的结构化报告。
Output / 强制输出结构
第一部分:文案构建逻辑与词频分析 (The Brain)
构建逻辑与方法论: 拆解其标题、五点描述、详情描述的文本构建策略。他是基于"痛点触发"、"场景驱动"还是"参数压制"?使用了什么样的叙事模板?
词频情报: 提取 Listing 全文中的 Top 10 核心关键词,并分析这些词是如何在文案中进行权重分配与埋点的。
第二部分:视觉资产设计思路 (The Face)
设计方法论: 分析主图与 A+ 图片的整体设计风格与视觉定位(如:极简工业风、居家生活风等)。
视觉动线拆解: 逐一说明其不同图片(功能图、场景图、对比图等)分别表达了什么核心内容?视觉上有哪些抓人眼球的特点?
设计逻辑: 分析其构图、色彩心理学应用以及字体选型,识别其视觉上的差异化"钩子"。
第三部分:评论定量与定性分析 (The Voice)
量化数据概览:
- 明确分析的样本量(如:前 X 条评论)及当前总评分
- 统计好评(4-5星)与差评(1-3星)的数量及百分比
定性穿透分析:
- 优势聚类:用户评论中反复提到的优点及其触发场景
- 差评穿透:差评主要体现的核心问题(区分产品缺陷、描述不符或体验感差)
核心总结 (Top 3):
- 3 条核心优势(用户为何买他)
- 3 条核心痛点(用户为何退货/差评)
- 3 条改进建议(我该如何做得更好)
第四部分:市场维态与盲区扫描 (The Pulse)
市场表现: 分析其价格波动规律、评分稳定性以及在类目中的排名变动趋势。
QA 价值挖掘: 从问答区提取用户下单前的"最后一道心理防线"。
盲区扫描: 识别任何我们尚未察觉但具有威胁或机会的内容(如:特定背书、捆绑策略等)。
---
Required Output Format
最后请你输出以下内容,包括:
1. 产品标题 2. 价格 3. 评分 4. 详细的竞品分析报告内容
*不要出现任何#这类没有意义的符号**
跨境电商竞品分析Skill
<div align="center">
一款强大的多平台电商竞品分析 Claude Code Skill
Created By Buluslan
想了解更多最新AI行业动态,AI+电商/广告的行业实践方法,人与AI如何协作共生的思考,请关注公众号:【新西楼】
 
</div>
概述
这是一个 Claude Code Skill,可以自动分析多个电商平台(Amazon、Temu、Shopee等)的竞品数据,并生成全面的 AI 分析报告。
核心功能
- 多平台支持:Amazon(已启用)、Temu 和 Shopee(计划中)
- 批量处理:单次请求分析多个产品
- AI 驱动分析:四维度分析框架:
- 文案策略与关键词分析
- 视觉资产设计方法论
- 客户评论情感分析
- 市场定位与竞争情报
- 双格式输出:
- Google Sheets(结构化数据)
- Markdown 报告(详细分析)
- 错误隔离:单个产品失败不会中断批量处理
什么是 Claude Code Skill?
技能是 Claude Code AI 的"使用手册"。它通过提供结构化的提示词、脚本和配置,让 Claude 能够执行专业任务。
使用本技能,你只需要说:
"分析这些 Amazon 产品:B0C4YT8S6H, B08N5WRQ1Y, B0CLFH7CCV"
Claude 就会: 1. 从 Amazon 提取产品数据 2. 生成 AI 驱动的分析报告,主要包括:
- 基础信息:商品标题、价格、评分情况等。
- 内容分析:listing文案的方法论总结,高频卖点和内容亮点,TOP10高频关键词分析。
- 视觉分析:主图和A+的设计方法论总结,视觉动线拆解。
- 评论分析:统计评论数量和星级分布,并详细分析最新的评论内容,分别总结3条产品优势和缺陷,输出改进建议。
- 其他分析:包括asin排名和市场动态,Q&A中的高频问题分析,以及整体的分析总结。
4. 将结果输出到 Google Sheets 和 Markdown 文件
系统要求
必需的 API 密钥
| 服务 | 用途 | 费用 |
|---|---|---|
| Olostep API | 网页数据抓取 | 1000 次免费请求/月,之后 $0.002/次 |
| Google Gemini API | AI 分析 | ~$0.001/产品 |
可选的 API 密钥
| 服务 | 用途 |
|---|---|
| Google Sheets API | 将结果导出到 Google Sheets |
安装
步骤 1:安装技能
# 使用 npx skills(推荐)
npx skills add buluslan/ecommerce-competitor-analyzer
# 或手动克隆
git clone https://github.com/buluslan/ecommerce-competitor-analyzer.git
cp -r ecommerce-competitor-analyzer ~/.claude/skills/main-mode-skills/ecommerce-competitor-analyzer.skill步骤 2:配置环境变量
# 复制示例环境文件
cd ~/.claude/skills/main-mode-skills/ecommerce-competitor-analyzer.skill
cp .env.example .env
# 编辑 .env 并添加你的 API 密钥
nano .env添加你的 API 密钥:
OLOSTEP_API_KEY=your_olostep_api_key_here
GEMINI_API_KEY=your_gemini_api_key_here
GOOGLE_SHEETS_ID=your_google_sheets_id_here步骤 3:验证安装
# 列出已安装的技能
~/.claude/list-skills.sh使用方法
基础用法(单个产品)
在 Claude Code 中,只需说:
分析这个 Amazon 产品:B0C4YT8S6HClaude 会: 1. 从 Amazon 提取产品数据 2. 生成全面的分析报告 3. 保存到 Google Sheets(1行)+ Markdown 文件
批量分析(多个产品)
分析这些 Amazon 产品:
B0C4YT8S6H
B08N5WRQ1Y
B0CLFH7CCV或使用 URL:
分析这些产品:
https://amazon.com/dp/B0C4YT8S6H
https://amazon.com/dp/B08N5WRQ1Y输出格式
格式 1:Google Sheets(结构化数据)
| ASIN | 产品标题 | 价格 | 评分 | 文案分析 | 视觉分析 | 评论分析 | 市场分析 |
|---|---|---|---|---|---|---|---|
| B0C4YT8S6H | Samsung Galaxy Tab A9+ | $159.99 | 4.4 | [300字摘要] | [300字摘要] | [300字摘要] | [300字摘要] |
格式 2:Markdown 报告(详细分析)
# Amazon 竞品分析报告
## 产品 1:B0C4YT8S6H
### 基本信息
- 标题:Samsung Galaxy Tab A9+ Plus 11" 64GB Android Tablet
- 价格:$159.99
- 评分:4.4/5
### 文案策略与关键词分析
[完整分析内容...]
### 视觉资产设计方法论
[完整分析内容...]
### 客户评论分析
[完整分析内容...]
### 市场定位与竞争情报
[完整分析内容...]配置
Google Sheets 设置(可选)
如果要将结果导出到 Google Sheets:
1. 创建 Google Cloud 项目
- 访问 Google Cloud Console
- 创建新项目
2. 启用 Google Sheets API
- 导航到"API 和服务" > "库"
- 搜索"Google Sheets API"
- 点击"启用"
3. 创建 OAuth2 凭证
- 导航到"API 和服务" > "凭据"
- 点击"创建凭据" > "OAuth 客户端 ID"
- 应用类型:"桌面应用"
- 下载 JSON 凭证文件
4. 配置技能
- 复制 JSON 文件内容
- 粘贴到
.env中的GOOGLE_SHEETS_CREDENTIALS - 添加你的 Google Sheets ID 作为
GOOGLE_SHEETS_ID
高级设置
编辑 platforms.yaml 进行高级配置:
settings:
max_batch_size: 20 # 每批最大产品数
concurrency_limit: 5 # 并发处理数
scraping_timeout: 120000 # 2分钟
analysis_timeout: 60000 # 1分钟项目结构
ecommerce-competitor-analyzer.skill/
├── SKILL.md # AI 指令手册
├── platforms.yaml # 平台配置
├── .env.example # 配置模板
├── scripts/ # 核心脚本
│ ├── detect-platform.js # 平台检测
│ ├── scrape-amazon.js # Amazon 爬虫
│ └── batch-processor.js # 批处理引擎
├── prompts/ # AI 提示词模板
│ ├── analysis-prompt-base.md # 基础分析框架
│ ├── analysis-prompt-amazon.md # Amazon 专用提示词
│ └── analysis-prompt-cross-platform.md # 跨平台对比
└── references/ # 文档
├── n8n-workflow-analysis.md # n8n 工作流参考
└── platform-differences.md # 平台对比API 密钥获取指南
1. Olostep API
1. 访问 https://olostep.com/ 2. 注册免费账户 3. 导航到 Dashboard > API Keys 4. 复制你的 API 密钥 5. 添加到 .env:OLOSTEP_API_KEY=your_key_here
费用:1000 次免费请求/月,之后 $0.002/次
2. Google Gemini API
1. 访问 https://aistudio.google.com/app/apikey 2. 点击"创建 API 密钥" 3. 复制密钥 4. 添加到 .env:GEMINI_API_KEY=your_key_here
费用:~$0.001/产品分析
3. Google Sheets API(可选)
参见上面的"Google Sheets 设置"部分。
故障排除
问题:"Olostep API key not found"
解决方案:确保你已从 .env.example 创建了 .env 文件并添加了你的 API 密钥。
问题:"Google Sheets authentication failed"
解决方案:确保 .env 中的 GOOGLE_SHEETS_CREDENTIALS 包含有效的 JSON(单行)。
问题:"Batch processing timeout"
解决方案:增加 .env 中的 SCRAPER_TIMEOUT 或减少批量大小。
问题:"部分产品失败但其他成功"
解决方案:这是预期行为。技能使用错误隔离机制 - 单个失败不会中断整批处理。检查输出报告中的失败项。
开发
运行测试
# 测试平台检测
node scripts/detect-platform.js https://amazon.com/dp/B0C4YT8S6H
# 测试爬虫
node scripts/scrape-amazon.js B0C4YT8S6H添加新平台
1. 在 platforms.yaml 中添加平台配置 2. 在 scripts/ 中创建爬虫脚本 3. 在 prompts/ 中创建分析提示词 4. 更新 scripts/detect-platform.js
贡献
欢迎贡献!请:
1. Fork 仓库 2. 创建功能分支 3. 进行更改 4. 提交 Pull Request
贡献方向
- [ ] 添加 Temu 平台支持
- [ ] 添加 Shopee 平台支持
- [ ] 改进错误处理
- [ ] 添加更多 AI 分析维度
- [ ] 创建 Excel 导出格式
- [ ] 添加 PDF 报告生成
许可证
本项目基于 MIT 许可证 - 详见 LICENSE 文件。
致谢
- 基于 n8n 工作流 v81 逻辑构建
- 使用 Olostep API 进行网页抓取
- 使用 Google Gemini API 进行 AI 分析
- 宝玉的 Skills 框架
支持
- 问题反馈:GitHub Issues
- 讨论交流:GitHub Discussions
- 联系Builder,请备注【github】:
<img width="717" height="714" alt="wechat_2025-10-17_173400_583" src="https://github.com/user-attachments/assets/7c406098-dcd9-4684-84bd-f0ed4213e95f" />
路线图
- [x] Amazon 平台支持
- [ ] Temu 平台支持
- [ ] Shopee 平台支持
- [ ] 跨平台对比
- [ ] 历史价格追踪
- [ ] 评论情感可视化
- [ ] 竞品价格提醒
- [ ] 自动每日分析
---
<div align="center">
专为跨境电商从业者打造 ❤️
</div>
n8n 工作流参考
来源:Amazon 竞品分析 n8n 工作流 (v81) 工作流 ID:N2Z4oEsWYFAFWDX3 位置:你的 n8n 工作流 URL(私有) 配置文件:工作流配置.json
---
概述
本文档记录了从 n8n 工作流实现中的关键模式、决策和经验,这些内容与 skill 实现相关。
---
工作流架构
节点流程 (v81)
Google Sheets 触发器
↓
Google Sheets (读取 ASINs)
↓
过滤器 (跳过已分析的)
↓
Olostep API (抓取100条评论)
↓
Set (解析数据)
↓
Google Gemini (AI 分析)
↓
Code (提取结构化数据) ← **v81 关键修复**
↓
Google Sheets (写入结果)关键节点
| 节点 | 用途 | 关键设置 |
|---|---|---|
| Filter | 跳过已分析的 ASIN | 检查"分析结果"列是否为空 |
| Olostep API | 抓取产品页面 | comments_number: 100 |
| Google Gemini | AI 分析 | 模型:gemini-3-flash-preview |
| Code - Extract | 解析 AI 响应 | 使用 $input.all() 进行批处理 |
---
关键修复与经验
问题 1:批处理不工作 (v79 → v81)
症状:工作流只处理了一个 ASIN,尽管从 Google Sheets 读取了多个。
根本原因:"Code - 提取结构化数据"节点使用了 $input.item.json 而不是 $input.all(),导致所有项目合并成一个。
证据(执行 #368):
Google Sheets: 3 个项目 ✅
Filter: 3 个项目 ✅
Olostep API: 3 个项目 ✅
Google Gemini: 3 个项目 ✅
Code - Extract: 1 个项目 ❌ (被合并!)解决方案 (v81):
// 之前(错误)
const item = $input.item.json;
// 只处理第一个项目
// 之后(正确)
const items = $input.all();
const results = items.map((item, index) => {
// 独立处理每个项目
return { json: { /* 提取的数据 */ } };
});
return results; // 返回所有结果关键模式:在 n8n Code 节点中进行批处理时,始终使用 $input.all() + .map()。
---
问题 2:表达式格式错误 (v71 → v72)
症状:工作流验证失败,提示"Invalid expression"错误。
根本原因:n8n 表达式必须使用 {{ }} 包裹,不能直接使用 JavaScript。
解决方案:
// 之前(错误)
{{ $('Set - 解析数据').item.json.markdownContent }}
// 之后(正确)
{{ $('Set - 解析数据').item.json.markdownContent }}关键模式:HTTP 节点中的所有 n8n 表达式必须使用双花括号。
---
Skill 实现的代码模式
1. 批处理模式
// 来自 n8n "Code - 提取结构化数据"节点 (v81)
const items = $input.all();
const results = items.map((item, index) => {
try {
// 从当前项目提取数据
const aiResponse = item.json.content?.parts?.[0]?.text || '';
// 获取该索引的上游数据
const upstreamData = $('Set - 解析数据').all();
const asin = upstreamData[index].json.asin;
// 返回结构化结果
return {
json: {
asin: asin,
extractedTitle: extractTitle(aiResponse),
extractedPrice: extractPrice(aiResponse),
extractedRating: extractRating(aiResponse)
}
};
} catch (error) {
// 错误隔离:单个失败不会停止批处理
return {
json: {
asin: 'unknown',
error: 'Processing failed'
}
};
}
});
return results;2. 正则提取模式
// 标题提取(多种备用模式)
const titlePatterns = [
/产品标题[::]+([^\n]+)/,
/Title[::]+([^\n]+)/
];
let title = '未知'; // 默认值
for (const pattern of titlePatterns) {
const match = aiResponse.match(pattern);
if (match) {
title = match[1].trim();
break;
}
}
// 价格提取
const pricePatterns = [
/价格[::]+[^0-9]*([0-9]+\.?[0-9]*)/,
/Price[::]+[^0-9]*([0-9]+\.?[0-9]*)/
];
// 评分提取
const ratingPatterns = [
/评分[::]+[^0-9]*([0-9]+\.?[0-9]*)/,
/Rating[::]+[^0-9]*([0-9]+\.?[0-9]*)/
];3. 错误隔离模式
// 带错误隔离的处理
const items = $input.all();
const results = items.map((item, index) => {
try {
// 处理逻辑
const data = processData(item);
return { success: true, data };
} catch (error) {
// 返回错误结果而不是抛出异常
return { success: false, error: error.message };
}
});
// 继续处理所有结果(成功 + 失败)
return results;---
Olostep API 配置
请求格式
{
"url": "https://www.amazon.com/dp/B0C4YT8S6H",
"wait_time": 10,
"screenshot": false,
"extract_dynamic_content": true,
"comments_number": 100 // 关键:100条评论用于深度分析
}响应格式
{
"task_id": "string",
"markdown_content": "完整的页面内容(markdown格式)",
"html_content": "完整的页面 HTML"
}关键设置
| 参数 | 值 | 说明 |
|---|---|---|
comments_number | 100 | 设置为 100 以进行深度评论分析 |
wait_time | 10 | 允许页面完全加载 |
extract_dynamic_content | true | 捕获 JS 渲染的内容 |
---
Gemini AI 配置
模型
- 模型:
gemini-3-flash-preview - 原因:快速且性价比高的分析任务
提示词结构
提示词结构包含: 1. 角色:专家级 Amazon 运营总监 + 品牌策略师 2. 目标:4 维度的深度产品分析 3. 输出:结构化分析 + 提取的字段
4 维度分析框架
1. 文案构建逻辑与词频分析 (The Brain)
- 构建策略(痛点/场景/规格驱动)
- Top 10 关键词提取
2. 视觉资产设计思路 (The Face)
- 设计方法论
- 视觉流程分解
- 色彩心理学
3. 评论定量与定性分析 (The Voice)
- 定量概述
- 优势聚类
- 负面评论深入分析
- Top 3 洞察
4. 市场维态与盲区扫描 (The Pulse)
- 价格趋势
- Q&A 分析
- 盲区识别
---
Google Sheets 集成
表格结构
| 列 | 用途 |
|---|---|
| A (ASIN) | 产品标识符(输入) |
| B (分析结果) | 完整的 AI 分析(输出) |
| C (标题) | 提取的标题(输出) |
| D (价格) | 提取的价格(输出) |
| E (评分) | 提取的评分(输出) |
过滤逻辑
// 如果已分析则跳过
const isAnalyzed = $input.item.json.分析结果 !== '';
return isAnalyzed === false;---
版本历史
| 版本 | 日期 | 关键变更 |
|---|---|---|
| v81 | 2026-01-28 | 修复批处理(Code 节点) |
| v80 | 2026-01-28 | 添加100条评论抓取 |
| v79 | 2026-01-28 | 批处理尝试 |
| v73-v78 | 2026-01-28 | 结构化数据提取 |
| v71-v72 | 2026-01-28 | 表达式格式修复 |
---
Skill 实现检查清单
- [x] 使用
$input.all()+.map()进行批处理 - [x] 实现错误隔离(单个失败 ≠ 批处理失败)
- [x] 使用 n8n 的确切 Gemini 提示词(无修改)
- [x] 使用正则备用模式提取标题/价格/评分
- [x] Olostep API 抓取100条评论
- [x] Google Sheets 双输出(表格 + markdown)
- [x] 支持 ASIN 和 URL 输入格式
#!/usr/bin/env node
/**
* Manual Google Sheets OAuth2 Authorization
* Simpler alternative for users who prefer manual flow
*/
const fs = require('fs');
const path = require('path');
const readline = require('readline');
// Load environment variables
function loadEnv() {
const envPath = path.join(__dirname, '..', '.env');
const envContent = fs.readFileSync(envPath, 'utf8');
const lines = envContent.split('\n');
for (const line of lines) {
const trimmedLine = line.trim();
if (trimmedLine && !trimmedLine.startsWith('#')) {
const [key, ...valueParts] = trimmedLine.split('=');
const value = valueParts.join('=').trim();
if (key && value) {
process.env[key.trim()] = value;
}
}
}
}
async function main() {
console.log('\n' + '='.repeat(60));
console.log('🔐 Google Sheets OAuth2 Manual Authorization');
console.log('='.repeat(60) + '\n');
loadEnv();
const config = {
clientId: process.env.GOOGLE_SHEETS_CLIENT_ID,
clientSecret: process.env.GOOGLE_SHEETS_CLIENT_SECRET,
redirectUri: process.env.GOOGLE_SHEETS_REDIRECT_URI || 'http://localhost:8080',
sheetId: process.env.GOOGLE_SHEETS_ID_DEFAULT,
sheetName: process.env.GOOGLE_SHEET_NAME_DEFAULT || '工作表1',
gid: process.env.GOOGLE_SHEET_GID
};
if (!config.clientId || !config.clientSecret) {
console.error('❌ Google Sheets credentials not found in .env');
console.error('Please set GOOGLE_SHEETS_CLIENT_ID and GOOGLE_SHEETS_CLIENT_SECRET');
process.exit(1);
}
console.log('📋 Configuration:');
console.log(` Spreadsheet ID: ${config.sheetId}`);
console.log(` Sheet Name: ${config.sheetName}`);
if (config.gid) {
console.log(` GID: ${config.gid}`);
}
console.log();
// Generate auth URL
const crypto = require('crypto');
const state = crypto.randomBytes(16).toString('hex');
const scope = 'https://www.googleapis.com/auth/spreadsheets';
const params = new URLSearchParams({
client_id: config.clientId,
redirect_uri: config.redirectUri,
scope: scope,
response_type: 'code',
state: state,
access_type: 'offline',
prompt: 'consent'
});
const authUrl = `https://accounts.google.com/o/oauth2/v2/auth?${params.toString()}`;
console.log('🔗 Step 1: Open this URL in your browser:\n');
console.log(` ${authUrl}\n`);
console.log('⚠️ Important: Make sure your OAuth2 client has this redirect URI:');
console.log(` ${config.redirectUri}\n`);
console.log('📝 Step 2: After authorizing, you will be redirected to a page');
console.log(' that shows an error (this is normal).');
console.log(' Copy the "code" parameter from the URL.\n');
const rl = readline.createInterface({
input: process.stdin,
output: process.stdout
});
rl.question('Paste the authorization code here: ', async (code) => {
code = code.trim();
if (!code) {
console.log('\n❌ No code provided. Authorization canceled.');
rl.close();
process.exit(1);
}
console.log('\n🔄 Exchanging code for tokens...');
try {
const response = await fetch('https://oauth2.googleapis.com/token', {
method: 'POST',
headers: {
'Content-Type': 'application/x-www-form-urlencoded'
},
body: new URLSearchParams({
code: code,
client_id: config.clientId,
client_secret: config.clientSecret,
redirect_uri: config.redirectUri,
grant_type: 'authorization_code'
})
});
if (!response.ok) {
const error = await response.text();
throw new Error(`Token exchange failed: ${error}`);
}
const tokens = await response.json();
// Save tokens
const tokenPath = path.join(__dirname, '..', '.google-tokens.json');
const tokenData = {
...tokens,
expiry_date: Date.now() + (tokens.expires_in * 1000)
};
fs.writeFileSync(tokenPath, JSON.stringify(tokenData, null, 2));
console.log('✅ Authorization successful!\n');
console.log(' Tokens saved to: .google-tokens.json');
console.log(' Access Token: ' + tokens.access_token.substring(0, 20) + '...');
if (tokens.refresh_token) {
console.log(' Refresh Token: ' + tokens.refresh_token.substring(0, 20) + '...');
}
console.log('\n' + '='.repeat(60));
console.log('✅ You can now use the skill to write to Google Sheets!');
console.log('='.repeat(60) + '\n');
rl.close();
} catch (error) {
console.error('\n❌ Authorization failed:', error.message);
console.error('\nPossible issues:');
console.error('1. Invalid authorization code');
console.error('2. Redirect URI mismatch in OAuth2 client settings');
console.error('3. OAuth2 client does not have correct scopes');
rl.close();
process.exit(1);
}
});
}
main();
#!/usr/bin/env node
/**
* Google Sheets Service Account Setup Guide
*
* Service accounts are simpler than OAuth2 for server-to-server communication
* No user authorization required - just need to share the sheet with the service account email
*/
const fs = require('fs');
const path = require('path');
console.log('\n' + '='.repeat(70));
console.log('🔐 Google Sheets Service Account Setup');
console.log('='.repeat(70) + '\n');
console.log('Service accounts are recommended for server applications.\n');
console.log('📋 Steps to set up:\n');
console.log('1. Create a Service Account');
console.log(' - Go to: https://console.cloud.google.com/iam-admin/serviceaccounts');
console.log(' - Click "Create Service Account"');
console.log(' - Name: ecommerce-competitor-analyzer');
console.log(' - Click "Create and Continue"\n');
console.log('2. Grant Permissions');
console.log(' - Role: Editor (or Sheets Editor)\n');
console.log('3. Create Key');
console.log(' - Click on the service account');
console.log(' - Go to "Keys" tab');
console.log(' - Click "Add Key" → "Create New Key"');
console.log(' - Key type: JSON');
console.log(' - Download the JSON file\n');
console.log('4. Save the Key');
console.log(' - Rename the downloaded file to: service-account-key.json');
console.log(' - Move it to: ' + path.join(__dirname, '..'));
console.log(' - Or paste the content below:\n');
console.log('5. Share the Google Sheet');
console.log(' - Open your Google Sheet');
console.log(' - Click "Share" button');
console.log(' - Add the service account email (looks like: xxx@xxx.iam.gserviceaccount.com)');
console.log(' - Grant "Editor" permission\n');
console.log('6. Update .env file');
console.log(' - Add: GOOGLE_SHEETS_USE_SERVICE_ACCOUNT=true');
console.log(' - Remove or comment out: GOOGLE_SHEETS_CLIENT_ID/SECRET/REDIRECT_URI\n');
console.log('='.repeat(70));
console.log('✅ After completing these steps, the skill will use service account auth');
console.log('='.repeat(70) + '\n');
// Check if service account key already exists
const keyPath = path.join(__dirname, '..', 'service-account-key.json');
if (fs.existsSync(keyPath)) {
console.log('✅ Service account key found at: service-account-key.json');
console.log(' You can skip step 4!\n');
} else {
console.log('⚠️ Service account key not found');
console.log(' Please complete step 4 to create and save the key\n');
}
#!/usr/bin/env node
/**
* Google Sheets OAuth2 Authorization Script
* Run this script to authorize access to Google Sheets
*/
const fs = require('fs');
const path = require('path');
// Load environment variables
function loadEnv() {
const envPath = path.join(__dirname, '..', '.env');
if (!fs.existsSync(envPath)) {
console.error('❌ .env file not found');
process.exit(1);
}
const envContent = fs.readFileSync(envPath, 'utf8');
const lines = envContent.split('\n');
for (const line of lines) {
const trimmedLine = line.trim();
if (trimmedLine && !trimmedLine.startsWith('#')) {
const [key, ...valueParts] = trimmedLine.split('=');
const value = valueParts.join('=').trim();
if (key && value) {
process.env[key.trim()] = value;
}
}
}
}
// Check dependencies
function checkDependencies() {
try {
require('http');
require('crypto');
} catch (error) {
console.error('❌ Missing required dependencies');
console.error('Please ensure you are running Node.js 14+');
process.exit(1);
}
}
// Main authorization flow
async function main() {
console.log('\n' + '='.repeat(60));
console.log('🔐 Google Sheets OAuth2 Authorization');
console.log('='.repeat(60) + '\n');
loadEnv();
checkDependencies();
const { getAuthUrl, exchangeCodeForToken, config } = require('./google-sheets-writer.js');
// Check configuration
if (!config.clientId || !config.clientSecret) {
console.error('❌ Google Sheets credentials not found in .env');
console.error('Please set GOOGLE_SHEETS_CLIENT_ID and GOOGLE_SHEETS_CLIENT_SECRET');
process.exit(1);
}
console.log('📋 Configuration:');
console.log(` Spreadsheet ID: ${config.sheetId}`);
console.log(` Sheet Name: ${config.sheetName}`);
if (config.gid) {
console.log(` GID: ${config.gid}`);
}
console.log();
// Generate authorization URL
const { url, state } = getAuthUrl();
console.log('🔗 Step 1: Open this URL in your browser:\n');
console.log(` ${url}\n`);
console.log('⚠️ Note: If the redirect URI mismatch error occurs, update your OAuth2 client');
console.log(` to include this redirect URI: ${config.redirectUri}\n`);
// Start a simple HTTP server to handle callback
const http = require('http');
const server = http.createServer((req, res) => {
const url = new URL(req.url, `http://${req.headers.host}`);
if (url.pathname === '/callback') {
const code = url.searchParams.get('code');
const returnedState = url.searchParams.get('state');
if (returnedState !== state) {
res.writeHead(400);
res.end('Invalid state parameter');
console.error('❌ State mismatch');
server.close();
return;
}
if (code) {
console.log('\n✅ Received authorization code');
console.log('🔄 Exchanging for tokens...');
exchangeCodeForToken(code)
.then((tokens) => {
console.log('✅ Authorization successful!\n');
console.log(' Tokens saved to: .google-tokens.json');
console.log(' Access Token: ' + tokens.access_token.substring(0, 20) + '...');
if (tokens.refresh_token) {
console.log(' Refresh Token: ' + tokens.refresh_token.substring(0, 20) + '...');
}
console.log('\n' + '='.repeat(60));
console.log('✅ You can now use the skill to write to Google Sheets!');
console.log('='.repeat(60) + '\n');
res.writeHead(200, { 'Content-Type': 'text/html' });
res.end(`
<html>
<head>
<title>Authorization Successful</title>
<style>
body { font-family: Arial, sans-serif; text-align: center; padding: 50px; }
.success { color: #4CAF50; font-size: 24px; }
</style>
</head>
<body>
<div class="success">✅ Authorization Successful!</div>
<p>You can close this window and return to the terminal.</p>
</body>
</html>
`);
server.close();
setTimeout(() => process.exit(0), 1000);
})
.catch((error) => {
console.error('\n❌ Token exchange failed:', error.message);
res.writeHead(500, { 'Content-Type': 'text/html' });
res.end(`
<html>
<head><title>Authorization Failed</title></head>
<body>
<h1>❌ Authorization Failed</h1>
<p>${error.message}</p>
</body>
</html>
`);
server.close();
setTimeout(() => process.exit(1), 1000);
});
} else {
const error = url.searchParams.get('error');
console.error('\n❌ Authorization denied:', error);
res.writeHead(400, { 'Content-Type': 'text/html' });
res.end(`
<html>
<head><title>Authorization Failed</title></head>
<body>
<h1>❌ Authorization Failed</h1>
<p>Reason: ${error}</p>
</body>
</html>
`);
server.close();
setTimeout(() => process.exit(1), 1000);
}
} else {
res.writeHead(404);
res.end('Not found');
}
});
const PORT = 8081;
server.listen(PORT, () => {
console.log(`🌐 Local server started on http://localhost:${PORT}`);
console.log('⏳ Waiting for authorization callback...\n');
});
// Timeout after 5 minutes
setTimeout(() => {
console.log('\n⏱️ Authorization timeout (5 minutes)');
server.close();
process.exit(1);
}, 5 * 60 * 1000);
}
// Run the script
main().catch(error => {
console.error('❌ Error:', error);
process.exit(1);
});
#!/bin/bash
# Google Sheets OAuth2 配置指南
echo "============================================================"
echo "🔧 Google Sheets OAuth2 重定向 URI 配置指南"
echo "============================================================"
echo ""
echo "❌ 错误: redirect_uri_mismatch"
echo ""
echo "这是因为你的 OAuth2 客户端配置中没有添加对应的重定向 URI"
echo ""
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "📋 解决步骤:"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo ""
echo "1️⃣ 打开 Google Cloud Console:"
echo " https://console.cloud.google.com/apis/credentials"
echo ""
echo "2️⃣ 找到 OAuth 2.0 客户端 ID:"
echo " 994295017900-j3hrfe9f53fil0fe1ek7lbd1ao1df5es.apps.googleusercontent.com"
echo ""
echo "3️⃣ 点击 '编辑' (或点击铅笔图标)"
echo ""
echo "4️⃣ 在 '已授权的重定向 URI' 部分,点击 '添加 URI'"
echo ""
echo "5️⃣ 添加以下 URI(任选其一或全部):"
echo ""
echo " 推荐(最简单):"
echo " urn:ietf:wg:oauth:2.0:oob"
echo ""
echo " 或者(本地回调):"
echo " http://localhost:8081"
echo " http://localhost:8082"
echo ""
echo "6️⃣ 点击 '保存'"
echo ""
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "⏳ 等待几分钟后,重新运行授权脚本:"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo ""
echo " node scripts/auth-oob.js"
echo ""
echo "============================================================"
echo ""
echo "💡 提示:"
echo " - 如果添加了 'urn:ietf:wg:oauth:2.0:oob',授权码会直接显示在浏览器中"
echo " - 如果添加了 'http://localhost:xxxx',需要本地服务器监听回调"
echo " - 修改后可能需要等待 1-2 分钟才能生效"
echo ""
#!/usr/bin/env node
/**
* 手动授权流程 - 从浏览器 URL 复制授权码
*/
const fs = require('fs');
const path = require('path');
const readline = require('readline');
function loadEnv() {
const envPath = path.join(__dirname, '..', '.env');
const envContent = fs.readFileSync(envPath, 'utf8');
const lines = envContent.split('\n');
for (const line of lines) {
const trimmedLine = line.trim();
if (trimmedLine && !trimmedLine.startsWith('#')) {
const [key, ...valueParts] = trimmedLine.split('=');
const value = valueParts.join('=').trim();
if (key && value) {
process.env[key.trim()] = value;
}
}
}
}
async function main() {
console.log('\n' + '='.repeat(70));
console.log('🔐 Google Sheets 手动授权流程');
console.log('='.repeat(70) + '\n');
loadEnv();
const config = {
clientId: process.env.GOOGLE_SHEETS_CLIENT_ID,
clientSecret: process.env.GOOGLE_SHEETS_CLIENT_SECRET,
redirectUri: process.env.GOOGLE_SHEETS_REDIRECT_URI || 'http://localhost:8081',
sheetId: process.env.GOOGLE_SHEETS_ID_DEFAULT
};
console.log('📋 配置信息:');
console.log(` 客户端 ID: ${config.clientId.substring(0, 20)}...`);
console.log(` 重定向 URI: ${config.redirectUri}`);
console.log(` 表格 ID: ${config.sheetId}\n`);
// Generate auth URL
const scope = 'https://www.googleapis.com/auth/spreadsheets';
const params = new URLSearchParams({
client_id: config.clientId,
redirect_uri: config.redirectUri,
scope: scope,
response_type: 'code',
access_type: 'offline',
prompt: 'consent'
});
const authUrl = `https://accounts.google.com/o/oauth2/v2/auth?${params.toString()}`;
console.log('━'.repeat(70));
console.log('📍 步骤 1:在浏览器中打开以下 URL');
console.log('━'.repeat(70));
console.log(`\n${authUrl}\n`);
console.log('💡 提示:选中 URL 后按 Cmd+C 复制\n');
console.log('━'.repeat(70));
console.log('📍 步骤 2:授权后处理');
console.log('━'.repeat(70));
console.log(`
1. 登录你的 Google 账号
2. 点击"允许"授权应用访问 Google Sheets
3. 浏览器会尝试跳转到 ${config.redirectUri}
4. 你会看到类似 "无法访问此网站" 或页面不存在(这是正常的!)
5. ⚠️ **重要**:复制浏览器地址栏中的完整 URL
6. 里面包含授权码(code 参数)
示例 URL:
${config.redirectUri}/?code=4/0Axxxxxxxxxxxx&scope=...
↑
这部分就是授权码
`);
const rl = readline.createInterface({
input: process.stdin,
output: process.stdout
});
rl.question('\n🔑 粘贴完整的回调 URL (或只粘贴授权码): ', async (input) => {
input = input.trim();
if (!input) {
console.log('\n❌ 未输入授权信息。授权已取消。');
rl.close();
process.exit(1);
}
// Extract code from URL or use input directly
let code = input;
if (input.includes('code=')) {
const match = input.match(/[?&]code=([^&]+)/);
if (match) {
code = match[1];
}
}
if (code.length < 10) {
console.log('\n❌ 授权码太短,可能不完整。');
rl.close();
process.exit(1);
}
console.log('\n⏳ 正在交换令牌...');
console.log(` 授权码: ${code.substring(0, 20)}...\n`);
try {
const response = await fetch('https://oauth2.googleapis.com/token', {
method: 'POST',
headers: {
'Content-Type': 'application/x-www-form-urlencoded'
},
body: new URLSearchParams({
code: code,
client_id: config.clientId,
client_secret: config.clientSecret,
redirect_uri: config.redirectUri,
grant_type: 'authorization_code'
})
});
if (!response.ok) {
const errorText = await response.text();
let errorMsg = `令牌交换失败 (${response.status})`;
try {
const errorJson = JSON.parse(errorText);
if (errorJson.error) {
errorMsg = `错误: ${errorJson.error}`;
if (errorJson.error_description) {
errorMsg += `\n${errorJson.error_description}`;
}
}
} catch (e) {
errorMsg += `\n${errorText}`;
}
throw new Error(errorMsg);
}
const tokens = await response.json();
// Save tokens
const tokenPath = path.join(__dirname, '..', '.google-tokens.json');
const tokenData = {
...tokens,
expiry_date: Date.now() + (tokens.expires_in * 1000)
};
fs.writeFileSync(tokenPath, JSON.stringify(tokenData, null, 2));
console.log('━'.repeat(70));
console.log('✅ 授权成功!');
console.log('━'.repeat(70));
console.log(`\n📁 令牌已保存: ${tokenPath}`);
console.log(`\n🔑 访问令牌: ${tokens.access_token.substring(0, 40)}...`);
if (tokens.refresh_token) {
console.log(`🔄 刷新令牌: ${tokens.refresh_token.substring(0, 40)}...`);
}
console.log('\n━'.repeat(70));
console.log('🎉 现在可以使用 skill 写入 Google Sheets 了!');
console.log('━'.repeat(70));
console.log('\n🧪 测试命令:');
console.log(' node scripts/test-skill.js B08LNY11RK');
console.log('\n📊 结果将写入:');
console.log(` https://docs.google.com/spreadsheets/d/${config.sheetId}\n`);
rl.close();
process.exit(0);
} catch (error) {
console.error('\n' + '━'.repeat(70));
console.error('❌ 授权失败');
console.error('━'.repeat(70));
console.error(`\n${error.message}\n`);
console.error('💡 常见问题:');
console.error('1. 授权码已过期(重新获取)');
console.error('2. 重定向 URI 未在 Google Cloud Console 中配置');
console.error(` 需要添加: ${config.redirectUri}`);
console.error('3. 授权码格式不正确\n');
rl.close();
process.exit(1);
}
});
}
main();
#!/usr/bin/env node
/**
* Out-of-Band Google Sheets Authorization
* Authorization code will be displayed in the browser - just copy and paste
*/
const fs = require('fs');
const path = require('path');
const readline = require('readline');
function loadEnv() {
const envPath = path.join(__dirname, '..', '.env');
const envContent = fs.readFileSync(envPath, 'utf8');
const lines = envContent.split('\n');
for (const line of lines) {
const trimmedLine = line.trim();
if (trimmedLine && !trimmedLine.startsWith('#')) {
const [key, ...valueParts] = trimmedLine.split('=');
const value = valueParts.join('=').trim();
if (key && value) {
process.env[key.trim()] = value;
}
}
}
}
async function main() {
console.log('\n' + '='.repeat(70));
console.log('🔐 Google Sheets 授权 (复制粘贴方式)');
console.log('='.repeat(70) + '\n');
loadEnv();
console.log('⚠️ 首次使用?请确保在 Google Cloud Console 添加此重定向 URI:');
console.log(' urn:ietf:wg:oauth:2.0:oob\n');
const config = {
clientId: process.env.GOOGLE_SHEETS_CLIENT_ID,
clientSecret: process.env.GOOGLE_SHEETS_CLIENT_SECRET,
redirectUri: 'urn:ietf:wg:oauth:2.0:oob',
sheetId: process.env.GOOGLE_SHEETS_ID_DEFAULT
};
// Generate auth URL
const scope = 'https://www.googleapis.com/auth/spreadsheets';
const params = new URLSearchParams({
client_id: config.clientId,
redirect_uri: config.redirectUri,
scope: scope,
response_type: 'code',
access_type: 'offline',
prompt: 'consent'
});
const authUrl = `https://accounts.google.com/o/oauth2/v2/auth?${params.toString()}`;
console.log('📋 步骤 1:复制以下 URL 并在浏览器中打开\n');
console.log('─'.repeat(70));
console.log(authUrl);
console.log('─'.repeat(70));
console.log('\n💡 提示:点击上方 URL 会自动复制(取决于终端)\n');
console.log('📝 步骤 2:完成授权后');
console.log(' - 浏览器会显示一个授权码');
console.log(' - 复制这个授权码\n');
const rl = readline.createInterface({
input: process.stdin,
output: process.stdout
});
rl.question('🔑 粘贴授权码: ', async (code) => {
code = code.trim();
if (!code || code.length < 10) {
console.log('\n❌ 授权码无效。授权已取消。');
rl.close();
process.exit(1);
}
console.log('\n⏳ 正在交换令牌...');
try {
const response = await fetch('https://oauth2.googleapis.com/token', {
method: 'POST',
headers: {
'Content-Type': 'application/x-www-form-urlencoded'
},
body: new URLSearchParams({
code: code,
client_id: config.clientId,
client_secret: config.clientSecret,
redirect_uri: config.redirectUri,
grant_type: 'authorization_code'
})
});
if (!response.ok) {
const errorText = await response.text();
let errorMsg = `令牌交换失败: ${response.status}`;
try {
const errorJson = JSON.parse(errorText);
if (errorJson.error) {
errorMsg = `错误: ${errorJson.error}`;
if (errorJson.error_description) {
errorMsg += `\n说明: ${errorJson.error_description}`;
}
}
} catch (e) {
// Not JSON, use text
}
throw new Error(errorMsg);
}
const tokens = await response.json();
// Save tokens
const tokenPath = path.join(__dirname, '..', '.google-tokens.json');
const tokenData = {
...tokens,
expiry_date: Date.now() + (tokens.expires_in * 1000)
};
fs.writeFileSync(tokenPath, JSON.stringify(tokenData, null, 2));
console.log('\n' + '='.repeat(70));
console.log('✅ 授权成功!');
console.log('='.repeat(70));
console.log('\n📁 令牌已保存到:');
console.log(` ${tokenPath}`);
console.log('\n🔑 访问令牌:');
console.log(' ' + tokens.access_token.substring(0, 40) + '...');
if (tokens.refresh_token) {
console.log('\n🔄 刷新令牌:');
console.log(' ' + tokens.refresh_token.substring(0, 40) + '...');
}
console.log('\n' + '='.repeat(70));
console.log('🎉 现在可以使用 skill 写入 Google Sheets 了!');
console.log('='.repeat(70));
console.log('\n🧪 测试一下:');
console.log(' node scripts/test-skill.js B08LNY11RK');
console.log('\n📊 结果将写入到:');
console.log(` https://docs.google.com/spreadsheets/d/${config.sheetId}\n`);
rl.close();
process.exit(0);
} catch (error) {
console.error('\n' + '='.repeat(70));
console.error('❌ 授权失败');
console.error('='.repeat(70));
console.error(`\n错误: ${error.message}\n`);
console.error('💡 可能的问题:');
console.error('1. 授权码无效或已过期(重新获取)');
console.error('2. 重定向 URI 未在 Google Cloud Console 中配置');
console.error(' 请添加: urn:ietf:wg:oauth:2.0:oob');
console.error('3. OAuth 客户端配置错误\n');
rl.close();
process.exit(1);
}
});
}
main();
#!/usr/bin/env node
/**
* Simple Manual Google Sheets Authorization
* Copy authorization code from browser URL
*/
const fs = require('fs');
const path = require('path');
const readline = require('readline');
// Load environment variables
function loadEnv() {
const envPath = path.join(__dirname, '..', '.env');
const envContent = fs.readFileSync(envPath, 'utf8');
const lines = envContent.split('\n');
for (const line of lines) {
const trimmedLine = line.trim();
if (trimmedLine && !trimmedLine.startsWith('#')) {
const [key, ...valueParts] = trimmedLine.split('=');
const value = valueParts.join('=').trim();
if (key && value) {
process.env[key.trim()] = value;
}
}
}
}
async function main() {
console.log('\n' + '='.repeat(70));
console.log('🔐 Google Sheets 手动授权');
console.log('='.repeat(70) + '\n');
loadEnv();
const config = {
clientId: process.env.GOOGLE_SHEETS_CLIENT_ID,
clientSecret: process.env.GOOGLE_SHEETS_CLIENT_SECRET,
redirectUri: process.env.GOOGLE_SHEETS_REDIRECT_URI || 'urn:ietf:wg:oauth:2.0:oob',
sheetId: process.env.GOOGLE_SHEETS_ID_DEFAULT
};
console.log('📋 配置:');
console.log(` Spreadsheet ID: ${config.sheetId}\n`);
// Generate auth URL using "out of band" redirect URI
const scope = 'https://www.googleapis.com/auth/spreadsheets';
const params = new URLSearchParams({
client_id: config.clientId,
redirect_uri: config.redirectUri,
scope: scope,
response_type: 'code',
access_type: 'offline',
prompt: 'consent'
});
const authUrl = `https://accounts.google.com/o/oauth2/v2/auth?${params.toString()}`;
console.log('🔗 步骤 1:在浏览器中打开以下 URL:\n');
console.log(` ${authUrl}\n`);
console.log('📝 步骤 2:授权后,会显示一个授权码');
console.log(' 复制这个授权码\n');
const rl = readline.createInterface({
input: process.stdin,
output: process.stdout
});
rl.question('粘贴授权码: ', async (code) => {
code = code.trim();
if (!code) {
console.log('\n❌ 未提供授权码。授权已取消。');
rl.close();
process.exit(1);
}
console.log('\n🔄 正在交换令牌...');
try {
const response = await fetch('https://oauth2.googleapis.com/token', {
method: 'POST',
headers: {
'Content-Type': 'application/x-www-form-urlencoded'
},
body: new URLSearchParams({
code: code,
client_id: config.clientId,
client_secret: config.clientSecret,
redirect_uri: config.redirectUri,
grant_type: 'authorization_code'
})
});
if (!response.ok) {
const error = await response.text();
throw new Error(`令牌交换失败: ${error}`);
}
const tokens = await response.json();
// Save tokens
const tokenPath = path.join(__dirname, '..', '.google-tokens.json');
const tokenData = {
...tokens,
expiry_date: Date.now() + (tokens.expires_in * 1000)
};
fs.writeFileSync(tokenPath, JSON.stringify(tokenData, null, 2));
console.log('\n✅ 授权成功!\n');
console.log(' 令牌已保存到: .google-tokens.json');
console.log(' 访问令牌: ' + tokens.access_token.substring(0, 30) + '...');
if (tokens.refresh_token) {
console.log(' 刷新令牌: ' + tokens.refresh_token.substring(0, 30) + '...');
}
console.log('\n' + '='.repeat(70));
console.log('✅ 现在可以使用 skill 写入 Google Sheets 了!');
console.log('='.repeat(70) + '\n');
console.log('🧪 测试一下:');
console.log(' node scripts/test-skill.js B08LNY11RK\n');
rl.close();
process.exit(0);
} catch (error) {
console.error('\n❌ 授权失败:', error.message);
console.error('\n可能的问题:');
console.error('1. 授权码无效或已过期');
console.error('2. 客户端密钥不正确');
console.error('3. 重定向 URI 不匹配');
rl.close();
process.exit(1);
}
});
}
main();
/**
* Batch Processor for E-commerce Competitor Analysis
*
* Core processing engine that orchestrates:
* 1. Product scraping (Olostep API)
* 2. AI analysis (Gemini)
* 3. Structured output (Google Sheets + Markdown)
*
* Based on n8n workflow batch processing pattern (v81)
* Reference: 工作流配置.json - "Code - 提取结构化数据" node
*/
const { scrapeAmazonProduct, batchScrapeAmazon, extractASIN } = require('./scrape-amazon.js');
/**
* Parse Google Sheets specification from user input
* @param {string} userInput - User's text input
* @returns {Object} - Parsed sheet configuration
*/
function parseGoogleSheetsSpec(userInput) {
if (!userInput) {
// Return default from env
return {
sheetId: process.env.GOOGLE_SHEETS_ID_DEFAULT || null,
sheetName: process.env.GOOGLE_SHEET_NAME_DEFAULT || '工作表1',
source: 'default'
};
}
const input = userInput.toLowerCase();
// Pattern 1: Sheet ID (e.g., "Sheet ID: abc123" or "SheetID:abc123")
const sheetIdMatch = userInput.match(/sheet\s*id\s*[::]\s*([a-zA-Z0-9-_]+)/);
if (sheetIdMatch) {
return {
sheetId: sheetIdMatch[1],
sheetName: '工作表1',
source: 'explicit_id'
};
}
// Pattern 2: Google Sheets URL (e.g., "https://docs.google.com/spreadsheets/d/abc123")
const urlMatch = userInput.match(/docs\.google\.com\/spreadsheets\/d\/([a-zA-Z0-9-_]+)/);
if (urlMatch) {
return {
sheetId: urlMatch[1],
sheetName: '工作表1',
source: 'url'
};
}
// Pattern 3: Table name (e.g., "写入到表格\"我的竞品分析\"")
const nameMatch = userInput.match(/表格["""](.+?)["""]|表格\s*[::]\s*(.+)/);
if (nameMatch) {
return {
sheetId: null, // Will search by name
sheetName: nameMatch[1] || nameMatch[2],
source: 'name'
};
}
// Return default
return {
sheetId: process.env.GOOGLE_SHEETS_ID_DEFAULT || null,
sheetName: process.env.GOOGLE_SHEET_NAME_DEFAULT || '工作表1',
source: 'default'
};
}
/**
* Process all input items (matching n8n $input.all() pattern)
* @param {Array} items - Array of product identifiers (ASINs/URLs)
* @param {Object} options - Processing options
* @returns {Promise<Array>} - Processed results
*/
async function processBatch(items, options = {}) {
const {
platform = 'amazon',
onProgress = null, // Callback for progress updates
onError = null // Callback for error handling
} = options;
console.log(`[BatchProcessor] Starting batch processing: ${items.length} items`);
const results = [];
const errors = [];
// Process each item (matching n8n .map() pattern)
for (let index = 0; index < items.length; index++) {
const item = items[index];
try {
// Progress callback
if (onProgress) {
onProgress({
current: index + 1,
total: items.length,
item: item
});
}
console.log(`[BatchProcessor] Processing item ${index + 1}/${items.length}: ${item}`);
// Step 1: Scrape product data
const scrapeResult = await scrapeAmazonProduct(item);
if (!scrapeResult.success) {
throw new Error(scrapeResult.error || 'Scraping failed');
}
// Step 2: AI analysis
const analysisResult = await analyzeWithAI(scrapeResult);
if (!analysisResult.success) {
throw new Error(analysisResult.error || 'AI analysis failed');
}
// Step 3: Extract structured data
const extractedData = extractStructuredData(
analysisResult.content,
scrapeResult.asin
);
// Step 4: Format output
results.push({
success: true,
index: index,
input: item,
asin: scrapeResult.asin,
scraped: scrapeResult,
analysis: analysisResult,
extracted: extractedData
});
} catch (error) {
console.error(`[BatchProcessor] Error processing item ${index + 1}:`, error.message);
const errorResult = {
success: false,
index: index,
input: item,
error: error.message,
timestamp: new Date().toISOString()
};
results.push(errorResult);
errors.push(errorResult);
// Error callback
if (onError) {
onError(errorResult);
}
// Continue processing (error isolation pattern from n8n)
continue;
}
}
console.log(`[BatchProcessor] Batch complete: ${results.filter(r => r.success).length}/${results.length} succeeded`);
return {
results: results,
summary: {
total: results.length,
succeeded: results.filter(r => r.success).length,
failed: results.filter(r => !r.success).length,
timestamp: new Date().toISOString()
}
};
}
/**
* Analyze scraped content with AI (Gemini)
* @param {Object} scrapedData - Data from scraper
* @returns {Promise<Object>} - AI analysis result
*/
async function analyzeWithAI(scrapedData) {
// Import prompt template
const prompt = getAnalysisPrompt();
// Prepare content for AI
const content = prompt.replace('{{ PRODUCT_CONTENT }}', scrapedData.markdownContent);
try {
// Call Gemini API (using n8n's model: gemini-3-flash-preview)
const response = await fetch('https://generativelanguage.googleapis.com/v1beta/models/gemini-3-flash-preview:generateContent', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'x-goog-api-key': process.env.GEMINI_API_KEY || ''
},
body: JSON.stringify({
contents: [{
parts: [{
text: content
}]
}]
})
});
if (!response.ok) {
throw new Error(`Gemini API error: ${response.status}`);
}
const data = await response.json();
return {
success: true,
content: data.candidates?.[0]?.content?.parts?.[0]?.text || '',
model: 'gemini-3-flash-preview',
timestamp: new Date().toISOString()
};
} catch (error) {
return {
success: false,
error: error.message,
timestamp: new Date().toISOString()
};
}
}
/**
* Extract structured data from AI response
* Matching n8n "Code - 提取结构化数据" node logic (v81)
* @param {string} aiResponse - AI analysis text
* @param {string} asin - Product ASIN
* @returns {Object} - Extracted structured data
*/
function extractStructuredData(aiResponse, asin) {
// Initialize with defaults (matching n8n pattern)
let title = '未知';
let price = '未知';
let rating = '未知';
// Extract title
const titlePatterns = [
/产品标题[::]+([^\n]+)/,
/Title[::]+([^\n]+)/
];
for (const pattern of titlePatterns) {
const match = aiResponse.match(pattern);
if (match) {
title = match[1].trim();
break;
}
}
// Extract price
const pricePatterns = [
/价格[::]+[^0-9]*([0-9]+\.?[0-9]*)/,
/Price[::]+[^0-9]*([0-9]+\.?[0-9]*)/
];
for (const pattern of pricePatterns) {
const match = aiResponse.match(pattern);
if (match) {
price = match[1];
break;
}
}
// Extract rating
const ratingPatterns = [
/评分[::]+[^0-9]*([0-9]+\.?[0-9]*)/,
/Rating[::]+[^0-9]*([0-9]+\.?[0-9]*)/
];
for (const pattern of ratingPatterns) {
const match = aiResponse.match(pattern);
if (match) {
rating = match[1];
break;
}
}
return {
asin: asin,
title: title,
price: price,
rating: rating,
fullAnalysis: aiResponse
};
}
/**
* Format results for Google Sheets output
* @param {Array} results - Processed results
* @returns {Array} - Formatted rows for Google Sheets
*/
function formatForGoogleSheets(results) {
const rows = [];
// Header row
rows.push([
'ASIN',
'产品标题',
'价格',
'评分',
'文案分析摘要',
'视觉分析摘要',
'评论分析摘要',
'市场分析摘要'
]);
// Data rows (successful results only)
for (const result of results) {
if (!result.success) continue;
const extracted = result.extracted;
const analysis = result.analysis.content;
// Extract summaries from analysis (300 chars each)
rows.push([
extracted.asin,
extracted.title,
extracted.price,
extracted.rating,
extractSummary(analysis, '文案构建', 300),
extractSummary(analysis, '视觉资产', 300),
extractSummary(analysis, '评论', 300),
extractSummary(analysis, '市场', 300)
]);
}
return rows;
}
/**
* Format results for Markdown report
* @param {Array} results - Processed results
* @returns {string} - Complete Markdown report
*/
function formatMarkdownReport(results) {
const date = new Date().toISOString().split('T')[0];
const successful = results.filter(r => r.success);
let markdown = `# 亚马逊竞品分析报告\n\n`;
markdown += `## 分析概览\n\n`;
markdown += `- 分析产品数:${successful.length}\n`;
markdown += `- 分析时间:${date}\n`;
markdown += `- 成功率:${successful.length}/${results.length}\n\n`;
markdown += `---\n\n`;
for (let i = 0; i < successful.length; i++) {
const result = successful[i];
const extracted = result.extracted || {};
// Use top-level asin and analysis, with fallback to extracted
const asin = result.asin || extracted.asin || '未知';
const title = extracted.title || '未知';
const price = extracted.price || '未知';
const rating = extracted.rating || '未知';
const analysis = result.analysis || extracted.fullAnalysis || '暂无分析';
markdown += `## 产品 ${i + 1}: ${asin}\n\n`;
markdown += `### 基本信息\n`;
markdown += `- 标题:${title}\n`;
markdown += `- 价格:${price}\n`;
markdown += `- 评分:${rating}\n\n`;
markdown += `### 详细分析\n\n`;
markdown += `${analysis}\n\n`;
markdown += `---\n\n`;
}
// Failed items
const failed = results.filter(r => !r.success);
if (failed.length > 0) {
markdown += `## 处理失败的产品\n\n`;
for (const item of failed) {
markdown += `- **${item.input}**: ${item.error}\n`;
}
}
return markdown;
}
/**
* Extract summary section from analysis
* @param {string} analysis - Full analysis text
* @param {string} keyword - Section keyword
* @param {number} maxLength - Maximum length
* @returns {string} - Extracted summary
*/
function extractSummary(analysis, keyword, maxLength) {
// Find section containing keyword
const index = analysis.indexOf(keyword);
if (index === -1) {
return analysis.substring(0, maxLength) + '...';
}
// Extract around the keyword
const start = Math.max(0, index - 50);
const end = Math.min(analysis.length, index + maxLength);
let summary = analysis.substring(start, end).trim();
if (summary.length > maxLength) {
summary = summary.substring(0, maxLength) + '...';
}
return summary;
}
/**
* Get analysis prompt from file
* @returns {string} - Prompt template
*/
function getAnalysisPrompt() {
// This would read from prompts/analysis-prompt-base.md
// For now, return a placeholder
return `你是亚马逊竞品分析专家。请分析以下产品页面的内容:
{{ PRODUCT_CONTENT }}
# Role / 身份角色
你是一位拥有 10 年经验的"亚马逊顶级运营总监"和"品牌战略官"...
[Full prompt from analysis-prompt-base.md]`;
}
// Export functions
module.exports = {
processBatch,
parseGoogleSheetsSpec,
analyzeWithAI,
extractStructuredData,
formatForGoogleSheets,
formatMarkdownReport
};
#!/bin/bash
# GitHub Release 创建脚本
# 需要先创建 GitHub Personal Access Token
TOKEN="YOUR_GITHUB_TOKEN_HERE"
REPO="buluslan/ecommerce-competitor-analyzer"
TAG="v1.0.0"
TITLE="🚀 E-commerce Competitor Analyzer v1.0.0"
# 读取 Release Notes
RELEASE_BODY=$(cat RELEASE_NOTES.md)
# 创建 Release
curl -X POST \
-H "Authorization: token $TOKEN" \
-H "Accept: application/vnd.github.v3+json" \
https://api.github.com/repos/$REPO/releases \
-d "{
\"tag_name\": \"$TAG\",
\"target_commitish\": \"main\",
\"name\": \"$TITLE\",
\"body\": $(echo "$RELEASE_BODY" | jq -Rs .),
\"draft\": false,
\"prerelease\": false
}"
echo "✅ Release created: https://github.com/$REPO/releases/tag/$TAG"
#!/usr/bin/env node
/**
* Exchange authorization code for access token
*/
const fs = require('fs');
const path = require('path');
function loadEnv() {
const envPath = path.join(__dirname, '..', '.env');
const envContent = fs.readFileSync(envPath, 'utf8');
const lines = envContent.split('\n');
for (const line of lines) {
const trimmedLine = line.trim();
if (trimmedLine && !trimmedLine.startsWith('#')) {
const [key, ...valueParts] = trimmedLine.split('=');
const value = valueParts.join('=').trim();
if (key && value) {
process.env[key.trim()] = value;
}
}
}
}
async function main() {
const code = process.argv[2];
if (!code) {
console.error('❌ 请提供授权码');
console.error('用法: node exchange-token.js <授权码>');
process.exit(1);
}
console.log('\n⏳ 正在交换令牌...');
loadEnv();
const config = {
clientId: process.env.GOOGLE_SHEETS_CLIENT_ID,
clientSecret: process.env.GOOGLE_SHEETS_CLIENT_SECRET,
redirectUri: process.env.GOOGLE_SHEETS_REDIRECT_URI || 'http://localhost:8081'
};
try {
const response = await fetch('https://oauth2.googleapis.com/token', {
method: 'POST',
headers: {
'Content-Type': 'application/x-www-form-urlencoded'
},
body: new URLSearchParams({
code: code,
client_id: config.clientId,
client_secret: config.clientSecret,
redirect_uri: config.redirectUri,
grant_type: 'authorization_code'
})
});
if (!response.ok) {
const errorText = await response.text();
let errorMsg = `令牌交换失败 (${response.status})`;
try {
const errorJson = JSON.parse(errorText);
if (errorJson.error) {
errorMsg = `错误: ${errorJson.error}`;
if (errorJson.error_description) {
errorMsg += `\n${errorJson.error_description}`;
}
}
} catch (e) {
errorMsg += `\n${errorText}`;
}
throw new Error(errorMsg);
}
const tokens = await response.json();
// Save tokens
const tokenPath = path.join(__dirname, '..', '.google-tokens.json');
const tokenData = {
...tokens,
expiry_date: Date.now() + (tokens.expires_in * 1000)
};
fs.writeFileSync(tokenPath, JSON.stringify(tokenData, null, 2));
console.log('━'.repeat(70));
console.log('✅ 授权成功!');
console.log('━'.repeat(70));
console.log(`\n📁 令牌已保存: ${tokenPath}`);
console.log(`\n🔑 访问令牌: ${tokens.access_token.substring(0, 40)}...`);
if (tokens.refresh_token) {
console.log(`🔄 刷新令牌: ${tokens.refresh_token.substring(0, 40)}...`);
}
console.log('\n━'.repeat(70));
console.log('🎉 现在可以使用 skill 写入 Google Sheets 了!');
console.log('━'.repeat(70));
console.log('\n🧪 测试一下:');
console.log(' node scripts/test-skill.js B08LNY11RK');
console.log('\n📊 结果将写入到:');
console.log(` https://docs.google.com/spreadsheets/d/${process.env.GOOGLE_SHEETS_ID_DEFAULT}\n`);
} catch (error) {
console.error('\n' + '━'.repeat(70));
console.error('❌ 令牌交换失败');
console.error('━'.repeat(70));
console.error(`\n${error.message}\n`);
process.exit(1);
}
}
main();
/**
* Google Sheets Writer
* Handles OAuth2 and Service Account authentication
*
* Based on n8n Google Sheets integration
*/
const fs = require('fs');
const path = require('path');
const crypto = require('crypto');
// Configuration from environment
const config = {
useServiceAccount: process.env.GOOGLE_SHEETS_USE_SERVICE_ACCOUNT === 'true',
clientId: process.env.GOOGLE_SHEETS_CLIENT_ID,
clientSecret: process.env.GOOGLE_SHEETS_CLIENT_SECRET,
redirectUri: process.env.GOOGLE_SHEETS_REDIRECT_URI || 'http://localhost:8080',
sheetId: process.env.GOOGLE_SHEETS_ID_DEFAULT,
sheetName: process.env.GOOGLE_SHEET_NAME_DEFAULT || '工作表1',
gid: process.env.GOOGLE_SHEET_GID
};
// Token storage file
const tokenPath = path.join(__dirname, '..', '.google-tokens.json');
const serviceAccountKeyPath = path.join(__dirname, '..', 'service-account-key.json');
// Service account credentials (loaded if using service account)
let serviceAccountCredentials = null;
/**
* Generate OAuth2 authorization URL
*/
function getAuthUrl() {
const scope = 'https://www.googleapis.com/auth/spreadsheets';
const state = crypto.randomBytes(16).toString('hex');
const params = new URLSearchParams({
client_id: config.clientId,
redirect_uri: config.redirectUri,
scope: scope,
response_type: 'code',
state: state,
access_type: 'offline',
prompt: 'consent'
});
return {
url: `https://accounts.google.com/o/oauth2/v2/auth?${params.toString()}`,
state: state
};
}
/**
* Exchange authorization code for access token
*/
async function exchangeCodeForToken(code) {
const response = await fetch('https://oauth2.googleapis.com/token', {
method: 'POST',
headers: {
'Content-Type': 'application/x-www-form-urlencoded'
},
body: new URLSearchParams({
code: code,
client_id: config.clientId,
client_secret: config.clientSecret,
redirect_uri: config.redirectUri,
grant_type: 'authorization_code'
})
});
if (!response.ok) {
const error = await response.text();
throw new Error(`Token exchange failed: ${error}`);
}
const tokens = await response.json();
// Save tokens
saveTokens(tokens);
return tokens;
}
/**
* Refresh access token using refresh token
*/
async function refreshAccessToken() {
const tokens = loadTokens();
if (!tokens || !tokens.refresh_token) {
throw new Error('No refresh token available. Please re-authorize.');
}
const response = await fetch('https://oauth2.googleapis.com/token', {
method: 'POST',
headers: {
'Content-Type': 'application/x-www-form-urlencoded'
},
body: new URLSearchParams({
client_id: config.clientId,
client_secret: config.clientSecret,
refresh_token: tokens.refresh_token,
grant_type: 'refresh_token'
})
});
if (!response.ok) {
const error = await response.text();
throw new Error(`Token refresh failed: ${error}`);
}
const newTokens = await response.json();
// Update tokens (keep refresh_token if not returned)
const updatedTokens = {
...tokens,
access_token: newTokens.access_token,
expires_in: newTokens.expires_in,
expiry_date: Date.now() + (newTokens.expires_in * 1000)
};
saveTokens(updatedTokens);
return updatedTokens;
}
/**
* Load tokens from file
*/
function loadTokens() {
if (fs.existsSync(tokenPath)) {
return JSON.parse(fs.readFileSync(tokenPath, 'utf8'));
}
return null;
}
/**
* Save tokens to file
*/
function saveTokens(tokens) {
const tokenData = {
...tokens,
expiry_date: tokens.expiry_date || (Date.now() + (tokens.expires_in * 1000))
};
fs.writeFileSync(tokenPath, JSON.stringify(tokenData, null, 2));
}
/**
* Load service account credentials
*/
function loadServiceAccountCredentials() {
if (serviceAccountCredentials) {
return serviceAccountCredentials;
}
if (!fs.existsSync(serviceAccountKeyPath)) {
return null;
}
try {
serviceAccountCredentials = JSON.parse(fs.readFileSync(serviceAccountKeyPath, 'utf8'));
return serviceAccountCredentials;
} catch (error) {
console.error('Failed to load service account key:', error.message);
return null;
}
}
/**
* Get service account access token using JWT
*/
async function getServiceAccountToken() {
const credentials = loadServiceAccountCredentials();
if (!credentials) {
throw new Error('Service account key file not found. Please create service-account-key.json');
}
// JWT implementation (simplified - for production use google-auth-library)
const header = {
alg: 'RS256',
typ: 'JWT'
};
const now = Math.floor(Date.now() / 1000);
const payload = {
iss: credentials.client_email,
scope: 'https://www.googleapis.com/auth/spreadsheets',
aud: 'https://oauth2.googleapis.com/token',
iat: now,
exp: now + 3600
};
// For simplicity, we'll use a different approach - direct API call with service account
// This requires the JWT to be signed, which is complex without external libraries
// Alternative: Use OAuth2 flow with service account's client_id/client_secret
const response = await fetch('https://oauth2.googleapis.com/token', {
method: 'POST',
headers: {
'Content-Type': 'application/x-www-form-urlencoded'
},
body: new URLSearchParams({
grant_type: 'urn:ietf:params:oauth:grant-type:jwt-bearer',
assertion: createJWT(credentials, header, payload)
})
});
if (!response.ok) {
// JWT approach failed, try using refresh_token if available
// Service accounts can also use the OAuth2 flow with a pre-configured refresh token
throw new Error(`Service account authentication failed: ${response.status}`);
}
const data = await response.json();
return data.access_token;
}
/**
* Create JWT for service account (simplified - needs proper signing)
* For production, use google-auth-library
*/
function createJWT(credentials, header, payload) {
// This is a placeholder - proper JWT signing requires crypto library
// For now, return empty to trigger error and suggest using OAuth2
throw new Error('JWT signing requires additional library. Please use OAuth2 authentication or install google-auth-library');
}
/**
* Get valid access token (supports both OAuth2 and Service Account)
*/
async function getAccessToken() {
// Check if service account is enabled
if (config.useServiceAccount) {
console.log('Using service account authentication...');
return await getServiceAccountToken();
}
// Use OAuth2
let tokens = loadTokens();
if (!tokens) {
throw new Error('Not authenticated. Please run authorization first.');
}
// Check if token is expired or will expire soon
if (!tokens.expiry_date || tokens.expiry_date < Date.now() + 60000) {
console.log('Token expired, refreshing...');
tokens = await refreshAccessToken();
}
return tokens.access_token;
}
/**
* Write data to Google Sheets
* @param {Array} rows - Array of arrays (each row is an array of values)
* @param {Object} options - Options
*/
async function writeToGoogleSheets(rows, options = {}) {
const accessToken = await getAccessToken();
const spreadsheetId = options.sheetId || config.sheetId;
const sheetName = options.sheetName || config.sheetName;
const range = options.range || `${sheetName}!A1`;
// Prepare the request body
const requestBody = {
values: rows,
valueInputOption: options.valueInputOption || 'USER_ENTERED'
};
// Use append or update based on options
const useAppend = options.append !== false;
const endpoint = useAppend ? 'append' : 'update';
let url = `https://sheets.googleapis.com/v4/spreadsheets/${spreadsheetId}/values/${encodeURIComponent(range)}`;
if (useAppend) {
url += `:append?valueInputOption=${requestBody.valueInputOption}&insertDataOption=INSERT_ROWS`;
} else {
url += `?valueInputOption=${requestBody.valueInputOption}`;
}
const response = await fetch(url, {
method: 'POST',
headers: {
'Authorization': `Bearer ${accessToken}`,
'Content-Type': 'application/json'
},
body: JSON.stringify(requestBody)
});
if (!response.ok) {
const error = await response.text();
throw new Error(`Google Sheets API error: ${response.status} - ${error}`);
}
const result = await response.json();
console.log(`✅ Successfully wrote ${rows.length} rows to Google Sheets`);
console.log(` Spreadsheet: https://docs.google.com/spreadsheets/d/${spreadsheetId}`);
return result;
}
/**
* Batch update cells in Google Sheets
*/
async function batchUpdateToGoogleSheets(updates, options = {}) {
const accessToken = await getAccessToken();
const spreadsheetId = options.sheetId || config.sheetId;
const requestBody = {
valueInputOption: options.valueInputOption || 'USER_ENTERED',
data: updates.map(update => ({
range: update.range,
values: update.values
}))
};
const response = await fetch(
`https://sheets.googleapis.com/v4/spreadsheets/${spreadsheetId}/values:batchUpdate`,
{
method: 'POST',
headers: {
'Authorization': `Bearer ${accessToken}`,
'Content-Type': 'application/json'
},
body: JSON.stringify(requestBody)
}
);
if (!response.ok) {
const error = await response.text();
throw new Error(`Google Sheets batch update error: ${response.status} - ${error}`);
}
const result = await response.json();
console.log(`✅ Successfully batch updated ${updates.length} ranges`);
return result;
}
/**
* Find first empty row in sheet
*/
async function findFirstEmptyRow() {
const accessToken = await getAccessToken();
const spreadsheetId = config.sheetId;
const sheetName = config.sheetName;
const response = await fetch(
`https://sheets.googleapis.com/v4/spreadsheets/${spreadsheetId}/values/${encodeURIComponent(sheetName)}!A:A`,
{
headers: {
'Authorization': `Bearer ${accessToken}`
}
}
);
if (!response.ok) {
const error = await response.text();
throw new Error(`Google Sheets read error: ${response.status} - ${error}`);
}
const data = await response.json();
// Find first empty row (skip header)
if (data.values && data.values.length > 0) {
return data.values.length + 1;
}
return 2; // Start at row 2 (row 1 is header)
}
/**
* Write analysis results to Google Sheets
*/
async function writeAnalysisResults(results) {
// Prepare header row
const headerRow = [
'ASIN',
'产品标题',
'价格',
'评分',
'文案分析摘要',
'视觉分析摘要',
'评论分析摘要',
'市场分析摘要'
];
// Prepare data rows
const dataRows = [];
for (const result of results) {
if (!result.success) continue;
const extracted = result.extracted || {};
const analysis = result.analysis || '';
// Extract summaries (first 300 chars of each section)
const summary = extractSummaries(analysis);
dataRows.push([
extracted.asin || result.asin || '',
extracted.title || '未知',
extracted.price || '未知',
extracted.rating || '未知',
summary.brain || '',
summary.face || '',
summary.voice || '',
summary.pulse || ''
]);
}
const sheetName = config.sheetName;
const accessToken = await getAccessToken();
const spreadsheetId = config.sheetId;
// Append header first
try {
const headerResponse = await fetch(
`https://sheets.googleapis.com/v4/spreadsheets/${spreadsheetId}/values/${encodeURIComponent(sheetName)}!A1:append?valueInputOption=USER_ENTERED`,
{
method: 'POST',
headers: {
'Authorization': `Bearer ${accessToken}`,
'Content-Type': 'application/json'
},
body: JSON.stringify({
values: [headerRow]
})
}
);
// Header write result (ignore if already exists)
} catch (e) {
// Ignore header write error
}
// Then append data rows
const response = await fetch(
`https://sheets.googleapis.com/v4/spreadsheets/${spreadsheetId}/values/${encodeURIComponent(sheetName)}!A:A:append?valueInputOption=USER_ENTERED&insertDataOption=INSERT_ROWS`,
{
method: 'POST',
headers: {
'Authorization': `Bearer ${accessToken}`,
'Content-Type': 'application/json'
},
body: JSON.stringify({
values: dataRows
})
}
);
if (!response.ok) {
const errorText = await response.text();
throw new Error(`Failed to append data: ${response.status} - ${errorText}`);
}
const result = await response.json();
console.log(`✅ Successfully appended ${dataRows.length} rows to Google Sheets`);
console.log(` Updated range: ${result.updates.updatedRange}`);
console.log(` View: https://docs.google.com/spreadsheets/d/${spreadsheetId}`);
return result;
}
/**
* Extract summaries from analysis text
*/
function extractSummaries(analysis) {
const sections = {
brain: '',
face: '',
voice: '',
pulse: ''
};
// Find each section and extract first 300 chars
const sectionPatterns = [
{ name: 'brain', pattern: /第一部分[::]*.*?文案.*?\n([\s\S]*?)(?=第二部分|第三部分|$)/i },
{ name: 'face', pattern: /第二部分[::]*.*?视觉.*?\n([\s\S]*?)(?=第三部分|第四部分|$)/i },
{ name: 'voice', pattern: /第三部分[::]*.*?评论.*?\n([\s\S]*?)(?=第四部分|$)/i },
{ name: 'pulse', pattern: /第四部分[::]*.*?市场.*?\n([\s\S]*?)$/i }
];
for (const section of sectionPatterns) {
const match = analysis.match(section.pattern);
if (match) {
sections[section.name] = match[1].trim().substring(0, 300);
}
}
return sections;
}
/**
* Clear authentication
*/
function clearAuth() {
if (fs.existsSync(tokenPath)) {
fs.unlinkSync(tokenPath);
console.log('✅ Authentication cleared');
}
}
// Export functions
module.exports = {
getAuthUrl,
exchangeCodeForToken,
getAccessToken,
writeToGoogleSheets,
batchUpdateToGoogleSheets,
writeAnalysisResults,
findFirstEmptyRow,
clearAuth,
config
};
/**
* Olostep API Client with v1/v2 Support
* Supports both API versions for backward compatibility
*/
// API Endpoints
const API_ENDPOINTS = {
v1: 'https://api.olostep.com/v1/scrapes',
v2: 'https://api.olostep.com/v2/agent/web-agent'
};
/**
* Scrape Amazon product using Olostep API
* @param {string} asin - Amazon ASIN
* @param {object} options - Options
* @param {string} options.apiVersion - 'v1' or 'v2' (default: 'v2')
* @param {string} options.apiKey - Olostep API key
* @param {number} options.waitTime - Wait time for page load (v2 only)
* @param {number} options.comments - Number of comments to scrape
* @returns {Promise<object>} - Scraped data
*/
async function scrapeWithOlostep(asin, options = {}) {
const {
apiVersion = process.env.OLOSTEP_API_VERSION || 'v1', // Default to v1 for compatibility
apiKey = process.env.OLOSTEP_API_KEY,
waitTime = 10,
comments = 100,
domain = 'amazon.com'
} = options;
const url = `https://www.${domain}/dp/${asin}`;
// Choose endpoint based on version
const endpoint = API_ENDPOINTS[apiVersion];
if (!endpoint) {
throw new Error(`Invalid API version: ${apiVersion}. Use 'v1' or 'v2'`);
}
console.log(`📡 Using Olostep API ${apiVersion}`);
try {
if (apiVersion === 'v1') {
return await scrapeV1(endpoint, url, apiKey, comments);
} else {
return await scrapeV2(endpoint, url, apiKey, waitTime, comments);
}
} catch (error) {
console.error(`❌ Olostep API ${apiVersion} error:`, error.message);
// Auto-fallback: if v2 fails, try v1
if (apiVersion === 'v2' && options.autoFallback !== false) {
console.warn('⚠️ v2 failed, attempting fallback to v1...');
return await scrapeV1(API_ENDPOINTS.v1, url, apiKey, comments);
}
throw error;
}
}
/**
* Scrape using v1 API
* IMPORTANT: Do NOT use 'extract' parameter - it causes 0% accuracy (returns wrong products)
* Matches n8n working configuration: only send 'url' parameter
*/
async function scrapeV1(endpoint, url, apiKey, comments) {
const response = await fetch(endpoint, {
method: 'POST',
headers: {
'Authorization': `Bearer ${apiKey}`,
'Content-Type': 'application/json'
},
body: JSON.stringify({
url: url
// NO extract parameter - let Olostep auto-detect (matches n8n config)
// Using extract parameter causes wrong products to be returned
})
});
if (!response.ok) {
const errorText = await response.text();
throw new Error(`Olostep v1 error: ${response.status} - ${errorText}`);
}
const data = await response.json();
// v1 API returns markdown content directly
const markdownContent = data.markdown_content || data.content || JSON.stringify(data, null, 2);
return {
success: true,
apiVersion: 'v1',
markdownContent: markdownContent,
rawData: data
};
}
/**
* Scrape using v2 API
*/
async function scrapeV2(endpoint, url, apiKey, waitTime, comments) {
const response = await fetch(endpoint, {
method: 'POST',
headers: {
'Authorization': `Bearer ${apiKey}`,
'Content-Type': 'application/json'
},
body: JSON.stringify({
url: url,
wait_time: waitTime,
screenshot: false,
extract_dynamic_content: true,
comments_number: comments
})
});
if (!response.ok) {
const errorText = await response.text();
throw new Error(`Olostep v2 error: ${response.status} - ${errorText}`);
}
const data = await response.json();
// Return v2 response
return {
success: true,
apiVersion: 'v2',
markdownContent: data.markdown_content || data.html_content || '',
rawData: data
};
}
// convertV1ToMarkdown removed - v1 API returns markdown directly
// The 'extract' parameter caused accuracy issues (0%), so we now use raw markdown from Olostep
module.exports = {
scrapeWithOlostep,
API_ENDPOINTS
};
/**
* Amazon Product Scraper using Olostep API
*
* Based on n8n workflow "Olostep API" node configuration (v81)
* Reference: 工作流配置.json
*
* Features:
* - Scrapes Amazon product pages (title, price, rating, reviews, images)
* - Returns structured data (markdown)
* - Error handling with retry logic
*/
// Olostep API Configuration
const OLOSTEP_API_ENDPOINT = 'https://api.olostep.com/v1/scrapes';
const OLOSTEP_API_KEY = process.env.OLOSTEP_API_KEY || '';
// Amazon URL patterns
const AMAZON_PATTERNS = {
asin: /\/dp\/([A-Z0-9]{10})/i,
product_url: /amazon\.(com|co\.uk|de|es|fr|it|ca|co\.jp)\/.*\/([A-Z0-9]{10})/i
};
/**
* Extract ASIN from various input formats
* @param {string} input - ASIN, URL, or product identifier
* @returns {string|null} - Extracted ASIN or null
*/
function extractASIN(input) {
// Direct ASIN format (10 alphanumeric characters)
const asinPattern = /^[A-Z0-9]{10}$/i;
if (asinPattern.test(input.trim())) {
return input.trim().toUpperCase();
}
// Extract from URL
for (const pattern of Object.values(AMAZON_PATTERNS)) {
const match = input.match(pattern);
if (match) {
return match[1] ? match[1].toUpperCase() : match[2].toUpperCase();
}
}
return null;
}
/**
* Build Amazon product URL from ASIN
* @param {string} asin - Amazon product ASIN
* @param {string} domain - Amazon domain (default: amazon.com)
* @returns {string} - Full product URL
*/
function buildAmazonURL(asin, domain = 'amazon.com') {
return `https://www.${domain}/dp/${asin}`;
}
/**
* Scrape Amazon product using Olostep API
* @param {string} asin - Amazon product ASIN
* @param {Object} options - Scraping options
* @returns {Promise<Object>} - Scraped data
*/
async function scrapeAmazonProduct(asin, options = {}) {
const {
domain = 'amazon.com',
timeout = 120000, // 2 minutes timeout
retries = 3
} = options;
const url = buildAmazonURL(asin, domain);
// Validate ASIN
if (!asin) {
throw new Error('Invalid ASIN provided');
}
// Prepare API request (matching n8n HTTP Request node)
// Only use 'url' parameter - let Olostep auto-detect and scrape all content
const requestBody = {
url: url
};
// API call with retry logic
for (let attempt = 1; attempt <= retries; attempt++) {
try {
const response = await fetch(OLOSTEP_API_ENDPOINT, {
method: 'POST',
headers: {
'Authorization': `Bearer ${OLOSTEP_API_KEY}`,
'Content-Type': 'application/json'
},
body: JSON.stringify(requestBody)
});
if (!response.ok) {
throw new Error(`Olostep API error: ${response.status} ${response.statusText}`);
}
const data = await response.json();
// v1 API returns data.result.markdown_content
const markdownContent = data.result?.markdown_content ||
data.markdown_content || '';
// Validate response structure
if (!markdownContent) {
throw new Error('Invalid response: missing content');
}
// Return structured data matching n8n workflow
return {
success: true,
asin: asin,
url: url,
markdownContent: markdownContent,
timestamp: new Date().toISOString()
};
} catch (error) {
console.error(`Scraping attempt ${attempt}/${retries} failed:`, error.message);
if (attempt === retries) {
// All retries exhausted
return {
success: false,
asin: asin,
url: url,
error: error.message,
timestamp: new Date().toISOString()
};
}
// Exponential backoff
await new Promise(resolve => setTimeout(resolve, Math.pow(2, attempt) * 1000));
}
}
}
/**
* Batch scrape multiple Amazon products
* @param {string[]} inputs - Array of ASINs or URLs
* @param {Object} options - Scraping options
* @returns {Promise<Object[]>} - Array of results
*/
async function batchScrapeAmazon(inputs, options = {}) {
const results = [];
// Process in parallel with concurrency limit
const CONCURRENCY_LIMIT = 5;
const chunks = [];
for (let i = 0; i < inputs.length; i += CONCURRENCY_LIMIT) {
chunks.push(inputs.slice(i, i + CONCURRENCY_LIMIT));
}
for (const chunk of chunks) {
const chunkResults = await Promise.allSettled(
chunk.map(input => {
const asin = extractASIN(input);
if (!asin) {
return Promise.resolve({
success: false,
input: input,
error: 'Invalid ASIN or URL format'
});
}
return scrapeAmazonProduct(asin, options);
})
);
// Convert PromiseSettledResults to standard format
for (let i = 0; i < chunkResults.length; i++) {
const result = chunkResults[i];
if (result.status === 'fulfilled') {
results.push(result.value);
} else {
results.push({
success: false,
input: chunk[i],
error: result.reason?.message || 'Unknown error'
});
}
}
}
return results;
}
// Export for use in skill
module.exports = {
extractASIN,
buildAmazonURL,
scrapeAmazonProduct,
batchScrapeAmazon
};
// For Node.js ES modules
// export { extractASIN, buildAmazonURL, scrapeAmazonProduct, batchScrapeAmazon };
#!/usr/bin/env node
/**
* Test Script for E-commerce Competitor Analyzer Skill
* Tests the complete workflow: Scraping → AI Analysis → Google Sheets Write
*/
const fs = require('fs');
const path = require('path');
const { validateProductData } = require('./data-validator.js');
const { scrapeWithOlostep } = require('./olostep-client.js');
// Load environment variables
function loadEnv() {
const envPath = path.join(__dirname, '..', '.env');
const envContent = fs.readFileSync(envPath, 'utf8');
const lines = envContent.split('\n');
for (const line of lines) {
const trimmedLine = line.trim();
if (trimmedLine && !trimmedLine.startsWith('#')) {
const [key, ...valueParts] = trimmedLine.split('=');
const value = valueParts.join('=').trim();
if (key && value) {
process.env[key.trim()] = value;
}
}
}
}
// Test scraping function (with v1/v2 compatibility)
async function testScrape(asin) {
console.log(`\n🔍 Testing scrape for ASIN: ${asin}`);
try {
// Use compatible client with auto-fallback
const result = await scrapeWithOlostep(asin, {
apiVersion: process.env.OLOSTEP_API_VERSION || 'v1', // Default to v1
comments: 100
});
if (!result.success) {
throw new Error(result.error || 'Scraping failed');
}
console.log(`✅ Scrape successful with API ${result.apiVersion}: ${result.markdownContent.length} characters`);
// Extract basic data for validation
const extractedData = {
title: extractTitle(result.markdownContent),
price: extractPrice(result.markdownContent),
rating: extractRating(result.markdownContent)
};
// Validate scraped data
const validation = validateProductData(asin, extractedData);
if (!validation.isValid) {
console.error(`\n⚠️ Validation Failed for ASIN ${asin}:`);
validation.issues.forEach(issue => console.error(` ❌ ${issue}`));
validation.warnings.forEach(warning => console.warn(` ⚠️ ${warning}`));
console.error(` ${validation.summary}`);
} else {
console.log(`✅ Validation passed`);
}
if (validation.warnings.length > 0) {
console.warn(`\n⚠️ Validation Warnings for ASIN ${asin}:`);
validation.warnings.forEach(warning => console.warn(` ⚠️ ${warning}`));
}
return {
success: true,
data: result.rawData,
markdownContent: result.markdownContent,
apiVersion: result.apiVersion,
validation: {
passed: validation.isValid,
...validation
},
extractedData
};
} catch (error) {
console.error(`❌ Scrape failed: ${error.message}`);
return { success: false, error: error.message };
}
}
// Helper functions to extract data from markdown content
function extractTitle(markdown) {
// Try to find title in various markdown formats
const patterns = [
/^#\s+(.+)$/m, // # Title
/##\s+Title\s*\n\s*(.+)$/m, // ## Title\n content
/##\s+产品标题\s*\n\s*(.+)$/m,
];
for (const pattern of patterns) {
const match = markdown.match(pattern);
if (match && match[1]) {
return match[1].trim();
}
}
// Fallback: return first non-empty line
const lines = markdown.split('\n').filter(l => l.trim());
return lines[0] || 'Unknown Title';
}
function extractPrice(markdown) {
// Try to extract price from markdown
const patterns = [
/\$\s*(\d+\.?\d*)/, // $19.99
/价格[:\s]*[^\d]*([\d,]+\.?\d*)/i, // 价格:$19.99
/Price[:\s]*[^\d]*([\d,]+\.?\d*)/i,
];
for (const pattern of patterns) {
const match = markdown.match(pattern);
if (match && match[1]) {
return match[1].replace(/,/g, '');
}
}
return null;
}
function extractRating(markdown) {
// Try to extract rating from markdown
const patterns = [
/(\d\.?\d*)\s*[☆★]/, // 4.5 ⭐
/评分[:\s]*[^\d]*(\d\.?\d*)/i, // 评分:4.5
/Rating[:\s]*[^\d]*(\d\.?\d*)/i,
];
for (const pattern of patterns) {
const match = markdown.match(pattern);
if (match && match[1]) {
return match[1];
}
}
return null;
}
// Test AI analysis function
async function testAIAnalysis(markdownContent, asin) {
console.log(`\n🤖 Testing AI analysis for ASIN: ${asin}`);
const GEMINI_API_KEY = process.env.GEMINI_API_KEY;
// Load prompt template
const promptPath = path.join(__dirname, '..', 'prompts', 'analysis-prompt-base.md');
let promptTemplate = '';
try {
promptTemplate = fs.readFileSync(promptPath, 'utf8');
} catch (error) {
promptTemplate = `你是亚马逊竞品分析专家。请分析以下产品页面的内容:\n\n{{ PRODUCT_CONTENT }}`;
}
// Replace placeholder
const prompt = promptTemplate.replace('{{ PRODUCT_CONTENT }}', markdownContent.substring(0, 10000)); // Limit content length
try {
const response = await fetch(`https://generativelanguage.googleapis.com/v1beta/models/gemini-3-flash-preview:generateContent?key=${GEMINI_API_KEY}`, {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify({
contents: [{
parts: [{
text: prompt
}]
}]
})
});
if (!response.ok) {
throw new Error(`Gemini API error: ${response.status}`);
}
const data = await response.json();
const analysisText = data.candidates?.[0]?.content?.parts?.[0]?.text || '';
console.log(`✅ AI analysis successful: ${analysisText.length} characters`);
return { success: true, content: analysisText };
} catch (error) {
console.error(`❌ AI analysis failed: ${error.message}`);
return { success: false, error: error.message };
}
}
// Test Google Sheets write function
async function testGoogleSheetsWrite(asin, title, price, rating, analysis) {
console.log(`\n📊 Testing Google Sheets write for ASIN: ${asin}`);
try {
// Check if tokens exist
const tokenPath = path.join(__dirname, '..', '.google-tokens.json');
if (!fs.existsSync(tokenPath)) {
console.log(` ⚠️ Not authenticated yet`);
console.log(` Run: node scripts/auth-google-sheets.js`);
console.log(`✅ Google Sheets write skipped (not authenticated)`);
return { success: true, authenticated: false };
}
// Load the writer module
const { writeAnalysisResults, config } = require('./google-sheets-writer.js');
console.log(` Target Sheet: ${config.sheetId}`);
console.log(` Target GID: ${config.gid || 'default'}`);
console.log(` - ASIN: ${asin}`);
console.log(` - Title: ${title}`);
console.log(` - Price: ${price}`);
console.log(` - Rating: ${rating}`);
// Note: We'll write all results at the end, not per-item
console.log(`✅ Google Sheets ready to write`);
return { success: true, authenticated: true };
} catch (error) {
console.error(` ❌ Google Sheets error: ${error.message}`);
return { success: false, error: error.message };
}
}
// Extract structured data from AI response
function extractData(aiResponse) {
let title = '未知';
let price = '未知';
let rating = '未知';
// Title patterns (handle multiple formats including markdown bold)
const titlePatterns = [
/\*\*1\.\s*产品标题\*\*[^\n]*\n([^\n]+)/,
/\*\*产品标题\*\*[^\n]*\n([^\n]+)/,
/产品标题[:\s]+([^\n]+)/,
/Title[:\s]+([^\n]+)/,
/产品标题[::]+([^\n]+)/,
/Title[::]+([^\n]+)/,
/1\.\s*产品标题[^\n]*\n([^\n]+)/
];
for (const pattern of titlePatterns) {
const match = aiResponse.match(pattern);
if (match) {
title = match[1].trim();
break;
}
}
// Price patterns (handle multiple formats)
const pricePatterns = [
/\*\*2\.\s*价格\*\*[^\n]*\n[^0-9\$¥]*([0-9]+\.?[0-9]*)/,
/\*\*价格\*\*[^\n]*\n[^0-9\$¥]*([0-9]+\.?[0-9]*)/,
/价格[:\s]+[^0-9\$¥]*([0-9]+\.?[0-9]*)/,
/Price[:\s]+[^0-9\$¥]*([0-9]+\.?[0-9]*)/,
/价格[::]+[^0-9]*([0-9]+\.?[0-9]*)/,
/Price[::]+[^0-9]*([0-9]+\.?[0-9]*)/,
/2\.\s*价格[^\n]*\n[^0-9\$¥]*([0-9]+\.?[0-9]*)/
];
for (const pattern of pricePatterns) {
const match = aiResponse.match(pattern);
if (match) {
price = match[1];
break;
}
}
// Rating patterns (handle multiple formats)
const ratingPatterns = [
/\*\*3\.\s*评分\*\*[^\n]*\n[^0-9\.]*([0-9]+\.?[0-9]*)/,
/\*\*评分\*\*[^\n]*\n[^0-9\.]*([0-9]+\.?[0-9]*)/,
/评分[:\s]+[^0-9\.]*([0-9]+\.?[0-9]*)/,
/Rating[:\s]+[^0-9\.]*([0-9]+\.?[0-9]*)/,
/评分[::]+[^0-9]*([0-9]+\.?[0-9]*)/,
/Rating[::]+[^0-9]*([0-9]+\.?[0-9]*)/,
/3\.\s*评分[^\n]*\n[^0-9\.]*([0-9]+\.?[0-9]*)/
];
for (const pattern of ratingPatterns) {
const match = aiResponse.match(pattern);
if (match) {
rating = match[1];
break;
}
}
return { title, price, rating };
}
// Main test function
async function runTest(asins) {
console.log('='.repeat(60));
console.log('🧪 E-commerce Competitor Analyzer - Test Run');
console.log('='.repeat(60));
console.log(`\nTest ASINs: ${asins.join(', ')}`);
console.log(`Target Sheet: ${process.env.GOOGLE_SHEETS_ID_DEFAULT} (GID: ${process.env.GOOGLE_SHEET_GID || 'default'})`);
const results = [];
for (const asin of asins) {
console.log(`\n${'─'.repeat(60)}`);
console.log(`Processing: ${asin}`);
console.log('─'.repeat(60));
// Step 1: Scrape
const scrapeResult = await testScrape(asin);
if (!scrapeResult.success) {
results.push({ asin, success: false, error: 'Scrape failed' });
continue;
}
// Step 2: AI Analysis
const analysisResult = await testAIAnalysis(scrapeResult.markdownContent, asin);
if (!analysisResult.success) {
results.push({ asin, success: false, error: 'AI analysis failed' });
continue;
}
// Step 3: Extract Data
const extracted = extractData(analysisResult.content);
console.log(`\n📋 Extracted Data:`);
console.log(` Title: ${extracted.title}`);
console.log(` Price: ${extracted.price}`);
console.log(` Rating: ${extracted.rating}`);
// Step 4: Write to Google Sheets
await testGoogleSheetsWrite(asin, extracted.title, extracted.price, extracted.rating, analysisResult.content);
results.push({
asin,
success: true,
extracted,
analysis: analysisResult.content
});
}
// Summary
console.log(`\n${'='.repeat(60)}`);
console.log('📊 Test Summary');
console.log('='.repeat(60));
console.log(`Total: ${asins.length}`);
console.log(`Success: ${results.filter(r => r.success).length}`);
console.log(`Failed: ${results.filter(r => !r.success).length}`);
// Save results to file (append mode)
const outputDir = path.join(__dirname, '..', 'output');
if (!fs.existsSync(outputDir)) {
fs.mkdirSync(outputDir, { recursive: true });
}
const outputFile = path.join(outputDir, 'analysis-results.json');
// Load existing results or create new array
let existingResults = [];
if (fs.existsSync(outputFile)) {
try {
const existingData = fs.readFileSync(outputFile, 'utf8');
existingResults = JSON.parse(existingData);
console.log(`\n📂 Loaded ${existingResults.length} existing records`);
} catch (error) {
console.warn(`⚠️ Could not read existing file, starting fresh`);
}
}
// Append new results
const updatedResults = [...existingResults, ...results];
fs.writeFileSync(outputFile, JSON.stringify(updatedResults, null, 2));
console.log(`\n💾 Results saved to: ${outputFile}`);
console.log(` Total records: ${updatedResults.length}`);
// Generate and save Markdown report
const { formatMarkdownReport } = require('./batch-processor.js');
const markdownReport = formatMarkdownReport(results);
// Create reports directory if it doesn't exist
const reportsDir = path.join(__dirname, '..', 'reports');
if (!fs.existsSync(reportsDir)) {
fs.mkdirSync(reportsDir, { recursive: true });
}
const dateStr = new Date().toISOString().split('T')[0];
const markdownFile = path.join(reportsDir, `竞品分析-${dateStr}.md`);
fs.writeFileSync(markdownFile, markdownReport);
console.log(`\n📝 Markdown report saved to: ${markdownFile}`);
// Write to Google Sheets (if authenticated)
console.log(`\n${'='.repeat(60)}`);
console.log('📊 Writing to Google Sheets');
console.log('='.repeat(60));
try {
const { writeAnalysisResults, config } = require('./google-sheets-writer.js');
console.log(` Target Sheet: ${config.sheetId}`);
console.log(` Target GID: ${config.gid || 'default'}`);
await writeAnalysisResults(results);
console.log(`\n✅ Successfully wrote results to Google Sheets!`);
console.log(` View: https://docs.google.com/spreadsheets/d/${config.sheetId}`);
} catch (error) {
if (error.message.includes('Not authenticated')) {
console.log(`\n⚠️ Google Sheets not authenticated`);
console.log(` To enable, run: node scripts/auth-google-sheets.js`);
} else {
console.error(`\n❌ Google Sheets write failed: ${error.message}`);
}
}
}
// Run the test
(async () => {
loadEnv();
const asins = process.argv.slice(2);
if (asins.length === 0) {
console.log('Usage: node test-skill.js <ASIN1> <ASIN2> ...');
console.log('\nRunning with default test ASINs...');
await runTest(['B08LNY11RK', 'B0F5HFG1N8']);
} else {
await runTest(asins);
}
})();
Related skills
FAQ
What does ecommerce-competitor-analyzer do?
Multi-platform e-commerce competitor analysis skill that automatically scrapes product data from Amazon, Temu, Shopee and generates comprehensive analysis reports using AI. Use when you need to analyz
When should I use ecommerce-competitor-analyzer?
During build integrations work for ai & agent building.
Is ecommerce-competitor-analyzer safe to install?
Review the Security Audits panel on this listing before production use.