Opencli Rs Web Scraper

# opencli-rs Web Scraper & Site Fetcher

> Skill by [ara.so](https://ara.so) — Daily 2026 Skills collection.

opencli-rs is a single 4.7MB Rust binary that fetches real-time data from 55+ websites (Twitter/X, Reddit, YouTube, HackerNews, Bilibili, Zhihu, Xiaohongshu, and more) with one command. It reuses your browser session via a Chrome extension, supports AI-native API discovery, controls Electron desktop apps (Cursor, ChatGPT, Notion), and passes through to external CLIs (gh, docker, kubectl). Up to 12x faster and 10x less memory than the Node.js original.

---

## Installation

### One-line (macOS / Linux)
```bash
curl -fsSL https://raw.githubusercontent.com/nashsu/opencli-rs/main/scripts/install.sh | sh
```

### Windows (PowerShell)
```powershell
Invoke-WebRequest -Uri "https://github.com/nashsu/opencli-rs/releases/latest/download/opencli-rs-x86_64-pc-windows-msvc.zip" -OutFile opencli-rs.zip
Expand-Archive opencli-rs.zip -DestinationPath .
Move-Item opencli-rs.exe "$env:LOCALAPPDATA\Microsoft\WindowsApps\"
```

### Build from Source
```bash
git clone https://github.com/nashsu/opencli-rs.git
cd opencli-rs
cargo build --release
cp target/release/opencli-rs /usr/local/bin/
```

### Chrome Extension (required for Browser-mode commands)
1. Download `opencli-rs-chrome-extension.zip` from [GitHub Releases](https://github.com/nashsu/opencli-rs/releases/latest)
2. Extract to any directory
3. Open `chrome://extensions` → enable **Developer mode** → **Load unpacked** → select the extracted folder
4. Extension auto-connects to the opencli-rs daemon

### AI Agent Skill Install
```bash
npx skills add https://github.com/nashsu/opencli-rs-skill
```

---

## Command Modes

| Mode | Requirement | Examples |
|------|-------------|---------|
| **Public** | Nothing — calls public APIs directly | hackernews, devto, arxiv, wikipedia |
| **Browser** | Chrome + extension running | twitter, bilibili, reddit, zhihu |
| **Desktop** | Target Electron app running | cursor, chatgpt, notion, discord |

---

## Key Commands

### Global Flags
```bash
opencli-rs --help                        # list all commands
opencli-rs <site> --help                 # site-specific help
opencli-rs <site> <cmd> --format json    # output: table | json | yaml | csv | markdown
opencli-rs <site> <cmd> --limit 20       # limit results
opencli-rs doctor                        # run diagnostics
opencli-rs completion bash >> ~/.bashrc  # shell completions (bash/zsh/fish)
```

### Public Commands (no browser needed)
```bash
# Hacker News
opencli-rs hackernews top --limit 10
opencli-rs hackernews search "rust async" --limit 5
opencli-rs hackernews user pg

# arXiv
opencli-rs arxiv search "large language models" --limit 10
opencli-rs arxiv paper 2303.08774

# Wikipedia
opencli-rs wikipedia summary "Rust programming language"
opencli-rs wikipedia search "memory safety"
opencli-rs wikipedia random

# Stack Overflow
opencli-rs stackoverflow hot
opencli-rs stackoverflow search "tokio async runtime"

# Dev.to
opencli-rs devto top
opencli-rs devto tag rust

# Linux.do
opencli-rs linux-do hot
opencli-rs linux-do search "Rust"
```

### Browser Commands (Chrome extension required)
```bash
# Twitter/X
opencli-rs twitter search "rust lang" --limit 10
opencli-rs twitter trending
opencli-rs twitter timeline
opencli-rs twitter bookmarks
opencli-rs twitter profile elonmusk
opencli-rs twitter post "Hello from opencli-rs!"
o