Llm Public Opinion Analytics Assistant

# LLM-Based Intelligent Public Opinion Analytics Assistant

> Skill by [ara.so](https://ara.so) — Data Skills collection.

## Overview

This project is an intelligent public opinion analysis assistant that combines real-time data from **26 trending lists across 15 mainstream platforms** with large language model (LLM) analysis capabilities. It provides conversational hot search queries, topic-specific searches, topic clustering analysis, and sentiment analysis through a web interface. The system supports keyboard shortcuts for crawler control, multi-platform data retrieval with direct navigation, and multi-channel hot topic push notifications (email, WeChat, Enterprise WeChat, Telegram).

## Key Features

- **Multi-Platform Data Collection**: Crawls 26 trending lists from 15 platforms
- **LLM-Powered Analysis**: Topic clustering, sentiment analysis, and trend detection
- **Conversational Interface**: Natural language queries for data exploration
- **Video Content Analysis**: Extracts information even from video-based news
- **Multi-Channel Notifications**: Email, WeChat Work, Telegram bot push notifications
- **Crawler Control**: Quick start/stop via keyboard shortcuts
- **Database Storage**: MySQL-based data persistence

## Installation

### Prerequisites

**Browser Driver Setup** (Required for news detail extraction):

1. **Check browser version**:
   - Open Edge/Chrome → Settings → About
   - Note your version (e.g., `115.0.5790.102`)

2. **Download matching driver**:
   - Chrome: https://chromedriver.chromium.org/
   - Edge: https://developer.microsoft.com/en-us/microsoft-edge/tools/webdriver/

3. **Install driver**:
   ```bash
   # Linux/macOS
   sudo mv chromedriver /usr/local/bin/
   sudo chmod +x /usr/local/bin/chromedriver
   
   # Windows: Add driver directory to PATH
   # e.g., C:\WebDriver\chromedriver.exe
   ```

4. **Verify installation**:
   ```bash
   chromedriver --version
   ```

### Environment Setup

```bash
# Clone repository
git clone https://github.com/hmmnxkl/LLM-Based-Intelligent-Public-Opinion-Analytics-Assistant.git
cd LLM-Based-Intelligent-Public-Opinion-Analytics-Assistant

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt
```

### Database Configuration

1. **Install MySQL** (8.0+ recommended)

2. **Create database and tables**:
   ```python
   # Reference init.py for schema
   import mysql.connector
   
   conn = mysql.connector.connect(
       host='localhost',
       user='your_user',
       password='your_password'
   )
   cursor = conn.cursor()
   
   # Create database
   cursor.execute("CREATE DATABASE IF NOT EXISTS hotsearch_db CHARACTER SET utf8mb4")
   cursor.execute("USE hotsearch_db")
   
   # Create tables (see init.py for full schema)
   cursor.execute("""
   CREATE TABLE IF NOT EXISTS hot_searches (
       id INT AUTO_INCREMENT PRIMARY KEY,
       platform VARCHAR(50),
       title VARCHAR(500),
       url VARCHAR(1000),
       rank INT,
       heat_value VARCHAR(100),
       timestamp DATETIME,
       content TEXT,
       sentiment VARCHAR(50),
       INDEX idx_platform_timestamp (platform, timestamp)
   ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
   """)
   
   conn.commit()
   cursor.close()
   conn.close()
   ```

### Configuration Files

**Create `.env` file in