opencli-rs Web Scraper & Site Fetcher

opencli-rs 网页抓取与站点信息获取工具

Skill by ara.so — Daily 2026 Skills collection.

opencli-rs is a single 4.7MB Rust binary that fetches real-time data from 55+ websites (Twitter/X, Reddit, YouTube, HackerNews, Bilibili, Zhihu, Xiaohongshu, and more) with one command. It reuses your browser session via a Chrome extension, supports AI-native API discovery, controls Electron desktop apps (Cursor, ChatGPT, Notion), and passes through to external CLIs (gh, docker, kubectl). Up to 12x faster and 10x less memory than the Node.js original.

由ara.so开发的Skill工具 — 属于2026每日技能合集。

opencli-rs是一个仅4.7MB的Rust单二进制文件，只需一条命令就能从55+网站（Twitter/X、Reddit、YouTube、HackerNews、B站、知乎、小红书等）获取实时数据。它可通过Chrome扩展复用你的浏览器会话，支持原生AI API发现，能控制Electron桌面应用（Cursor、ChatGPT、Notion），还可透传调用外部CLI工具（gh、docker、kubectl）。相比基于Node.js的原版，速度提升最高12倍，内存占用减少10倍。

Installation

安装

One-line (macOS / Linux)

一键安装（macOS / Linux）

bash

curl -fsSL https://raw.githubusercontent.com/nashsu/opencli-rs/main/scripts/install.sh | sh

bash

curl -fsSL https://raw.githubusercontent.com/nashsu/opencli-rs/main/scripts/install.sh | sh

Windows (PowerShell)

Windows（PowerShell）

powershell

Invoke-WebRequest -Uri "https://github.com/nashsu/opencli-rs/releases/latest/download/opencli-rs-x86_64-pc-windows-msvc.zip" -OutFile opencli-rs.zip
Expand-Archive opencli-rs.zip -DestinationPath .
Move-Item opencli-rs.exe "$env:LOCALAPPDATA\Microsoft\WindowsApps\"

powershell

Invoke-WebRequest -Uri "https://github.com/nashsu/opencli-rs/releases/latest/download/opencli-rs-x86_64-pc-windows-msvc.zip" -OutFile opencli-rs.zip
Expand-Archive opencli-rs.zip -DestinationPath .
Move-Item opencli-rs.exe "$env:LOCALAPPDATA\Microsoft\WindowsApps\"

Build from Source

从源码构建

bash

git clone https://github.com/nashsu/opencli-rs.git
cd opencli-rs
cargo build --release
cp target/release/opencli-rs /usr/local/bin/

bash

git clone https://github.com/nashsu/opencli-rs.git
cd opencli-rs
cargo build --release
cp target/release/opencli-rs /usr/local/bin/

Chrome Extension (required for Browser-mode commands)

Chrome扩展（浏览器模式命令必需）

Download
```
opencli-rs-chrome-extension.zip
```
from GitHub Releases
Extract to any directory
Open
```
chrome://extensions
```
→ enable Developer mode → Load unpacked → select the extracted folder
Extension auto-connects to the opencli-rs daemon

从GitHub Releases下载
```
opencli-rs-chrome-extension.zip
```
解压到任意目录
打开
```
chrome://extensions
```
→ 开启开发者模式 → 加载已解压的扩展程序 → 选择解压后的文件夹
扩展会自动连接到opencli-rs守护进程

AI Agent Skill Install

AI Agent Skill安装

bash

npx skills add https://github.com/nashsu/opencli-rs-skill

bash

npx skills add https://github.com/nashsu/opencli-rs-skill

Command Modes

命令模式

Mode	Requirement	Examples
Public	Nothing — calls public APIs directly	hackernews, devto, arxiv, wikipedia
Browser	Chrome + extension running	twitter, bilibili, reddit, zhihu
Desktop	Target Electron app running	cursor, chatgpt, notion, discord

模式	要求	示例
公共模式	无限制 — 直接调用公开API	hackernews, devto, arxiv, wikipedia
浏览器模式	需运行Chrome及扩展	twitter, bilibili, reddit, zhihu
桌面模式	需运行目标Electron应用	cursor, chatgpt, notion, discord

Key Commands

核心命令

Global Flags

全局参数

bash

opencli-rs --help                        # list all commands
opencli-rs <site> --help                 # site-specific help
opencli-rs <site> <cmd> --format json    # output: table | json | yaml | csv | markdown
opencli-rs <site> <cmd> --limit 20       # limit results
opencli-rs doctor                        # run diagnostics
opencli-rs completion bash >> ~/.bashrc  # shell completions (bash/zsh/fish)

bash

opencli-rs --help                        # 列出所有命令
opencli-rs <site> --help                 # 查看站点专属帮助
opencli-rs <site> <cmd> --format json    # 输出格式：table | json | yaml | csv | markdown
opencli-rs <site> <cmd> --limit 20       # 限制结果数量
opencli-rs doctor                        # 运行诊断
opencli-rs completion bash >> ~/.bashrc  # 自动补全配置（bash/zsh/fish）

Public Commands (no browser needed)

公共模式命令（无需浏览器）

bash

undefined

bash

undefined

Hacker News

opencli-rs hackernews top --limit 10 opencli-rs hackernews search "rust async" --limit 5 opencli-rs hackernews user pg

arXiv

opencli-rs arxiv search "large language models" --limit 10 opencli-rs arxiv paper 2303.08774

Wikipedia

opencli-rs wikipedia summary "Rust programming language" opencli-rs wikipedia search "memory safety" opencli-rs wikipedia random

Stack Overflow

opencli-rs stackoverflow hot opencli-rs stackoverflow search "tokio async runtime"

Dev.to

opencli-rs devto top opencli-rs devto tag rust

Linux.do

opencli-rs linux-do hot opencli-rs linux-do search "Rust"

undefined

opencli-rs linux-do hot opencli-rs linux-do search "Rust"

undefined

Browser Commands (Chrome extension required)

浏览器模式命令（需Chrome扩展）

bash

undefined

bash

undefined

Twitter/X

opencli-rs twitter search "rust lang" --limit 10 opencli-rs twitter trending opencli-rs twitter timeline opencli-rs twitter bookmarks opencli-rs twitter profile elonmusk opencli-rs twitter post "Hello from opencli-rs!" opencli-rs twitter follow rustlang

Bilibili

B站

opencli-rs bilibili hot --limit 20 opencli-rs bilibili search "Rust教程" opencli-rs bilibili ranking opencli-rs bilibili feed opencli-rs bilibili download <video_url>

Reddit

opencli-rs reddit frontpage opencli-rs reddit popular opencli-rs reddit subreddit rust opencli-rs reddit search "async await" opencli-rs reddit upvote <post_id>

Zhihu

知乎

opencli-rs zhihu hot opencli-rs zhihu search "Rust 内存安全" opencli-rs zhihu question 12345

Xiaohongshu

小红书

opencli-rs xiaohongshu search "旅行攻略" opencli-rs xiaohongshu feed opencli-rs xiaohongshu user <user_id>

YouTube

opencli-rs youtube search "rust tutorial" opencli-rs youtube video <video_id> opencli-rs youtube transcript <video_id>

Weibo

微博

opencli-rs weibo hot opencli-rs weibo search "科技"

Douban

豆瓣

opencli-rs douban top250 opencli-rs douban search "三体" opencli-rs douban movie-hot

Medium

opencli-rs medium search "rust programming" opencli-rs medium feed opencli-rs medium user graydon_hoare

Xueqiu (stock)

雪球（股票）

opencli-rs xueqiu hot-stock opencli-rs xueqiu search "茅台" opencli-rs xueqiu stock SH600519

undefined

opencli-rs xueqiu hot-stock opencli-rs xueqiu search "茅台" opencli-rs xueqiu stock SH600519

undefined

Desktop App Control (Electron)

桌面应用控制（Electron）

bash

undefined

bash

undefined

Cursor IDE

opencli-rs cursor status opencli-rs cursor send "Refactor this function to use async/await" opencli-rs cursor read opencli-rs cursor ask "What does this code do?" opencli-rs cursor screenshot opencli-rs cursor extract-code opencli-rs cursor model

ChatGPT Desktop

ChatGPT桌面端

opencli-rs chatgpt new opencli-rs chatgpt send "Explain Rust lifetimes" opencli-rs chatgpt read

Notion

opencli-rs notion search "project notes" opencli-rs notion read <page_id> opencli-rs notion write <page_id> "New content" opencli-rs notion sidebar

Discord Desktop

Discord桌面端

opencli-rs discord-app channels opencli-rs discord-app send "Hello team" opencli-rs discord-app read

undefined

opencli-rs discord-app channels opencli-rs discord-app send "Hello team" opencli-rs discord-app read

undefined

External CLI Passthrough

外部CLI透传调用

bash

undefined

bash

undefined

GitHub CLI

opencli-rs gh repo list opencli-rs gh pr list opencli-rs gh issue create --title "Bug report"

Docker

opencli-rs docker ps opencli-rs docker images opencli-rs docker logs my-container

Kubernetes

opencli-rs kubectl get pods opencli-rs kubectl get services -n production opencli-rs kubectl logs <pod-name>

undefined

opencli-rs kubectl get pods opencli-rs kubectl get services -n production opencli-rs kubectl logs <pod-name>

undefined

AI Discovery Commands

AI发现命令

bash

undefined

bash

undefined

Explore a website's API surface

探索网站的API接口

opencli-rs explore https://example.com

Auto-detect authentication strategies

自动检测认证策略

opencli-rs cascade https://api.example.com/data

Auto-generate an adapter (YAML pipeline)

自动生成适配器（YAML流水线）

opencli-rs generate https://example.com --goal "hot posts"

Synthesize adapter from discovered API

从发现的API合成适配器

opencli-rs synthesize https://news.example.com

---

opencli-rs synthesize https://news.example.com

---

Output Formats

输出格式

bash

undefined

bash

undefined

Default: ASCII table

默认：ASCII表格

opencli-rs hackernews top --limit 5

JSON — great for piping to jq

JSON — 适合配合jq使用

opencli-rs hackernews top --limit 5 --format json | jq '.[].title'

YAML

opencli-rs hackernews top --format yaml

CSV — for spreadsheets

CSV — 适用于电子表格

opencli-rs hackernews top --format csv > hn_top.csv

Markdown — for docs

Markdown — 适用于文档

opencli-rs hackernews top --format markdown

---

opencli-rs hackernews top --format markdown

---

Declarative YAML Pipeline (Custom Adapters)

声明式YAML流水线（自定义适配器）

Add new site adapters without writing Rust code. Create a YAML file describing the scraping pipeline:

yaml

undefined

无需编写Rust代码即可添加新站点适配器。创建一个YAML文件描述抓取流水线：

yaml

undefined

~/.config/opencli-rs/adapters/my-site.yaml

name: my-site description: Fetch top posts from my-site base_url: https://api.my-site.com commands: top: description: Get top posts endpoint: /v1/posts/top method: GET params: limit: flag: --limit default: 10 query_param: count response: items_path: $.data.posts fields: - name: title path: $.title - name: url path: $.url - name: score path: $.points - name: author path: $.author.name


```bash

name: my-site description: Fetch top posts from my-site base_url: https://api.my-site.com commands: top: description: Get top posts endpoint: /v1/posts/top method: GET params: limit: flag: --limit default: 10 query_param: count response: items_path: $.data.posts fields: - name: title path: $.title - name: url path: $.url - name: score path: $.points - name: author path: $.author.name


```bash

Use the custom adapter

使用自定义适配器

opencli-rs my-site top --limit 20 opencli-rs my-site top --format json

---

opencli-rs my-site top --limit 20 opencli-rs my-site top --format json

---

Real-World Patterns

实际使用场景

Pipe JSON output to jq for filtering

配合jq过滤JSON输出

bash

undefined

bash

undefined

Get only titles from HN top stories

仅获取HN热门文章的标题

opencli-rs hackernews top --limit 20 --format json | jq -r '.[].title'

Get Twitter trending topics as plain list

获取Twitter热门话题的纯文本列表

opencli-rs twitter trending --format json | jq -r '.[].name'

Find Bilibili videos with >1M views

筛选B站播放量超100万的视频

opencli-rs bilibili ranking --format json | jq '[.[] | select(.view > 1000000)]'

undefined

opencli-rs bilibili ranking --format json | jq '[.[] | select(.view > 1000000)]'

undefined

Shell scripting

Shell脚本示例

bash

#!/bin/bash

bash

#!/bin/bash

Daily digest script

每日摘要脚本

echo "=== HackerNews Top 5 ===" opencli-rs hackernews top --limit 5 --format table

echo "" echo "=== Bilibili Trending ===" opencli-rs bilibili hot --limit 5 --format table

echo "" echo "=== Zhihu Hot ===" opencli-rs zhihu hot --limit 5 --format table

undefined

echo "=== HackerNews Top 5 ===" opencli-rs hackernews top --limit 5 --format table

echo "" echo "=== B站热门 ===" opencli-rs bilibili hot --limit 5 --format table

echo "" echo "=== 知乎热榜 ===" opencli-rs zhihu hot --limit 5 --format table

undefined

Export to file

导出到文件

bash

opencli-rs reddit popular --format csv > reddit_$(date +%Y%m%d).csv
opencli-rs hackernews top --format json > hn_top.json

bash

opencli-rs reddit popular --format csv > reddit_$(date +%Y%m%d).csv
opencli-rs hackernews top --format json > hn_top.json

Use in Rust project via subprocess

在Rust项目中通过子进程调用

rust

use std::process::Command;
use serde_json::Value;

fn fetch_hn_top(limit: u32) -> anyhow::Result<Vec<Value>> {
    let output = Command::new("opencli-rs")
        .args(["hackernews", "top", "--limit", &limit.to_string(), "--format", "json"])
        .output()?;

    let json: Vec<Value> = serde_json::from_slice(&output.stdout)?;
    Ok(json)
}

fn fetch_twitter_search(query: &str) -> anyhow::Result<Vec<Value>> {
    let output = Command::new("opencli-rs")
        .args(["twitter", "search", query, "--format", "json"])
        .output()?;

    if !output.status.success() {
        let err = String::from_utf8_lossy(&output.stderr);
        anyhow::bail!("opencli-rs error: {}", err);
    }

    let json: Vec<Value> = serde_json::from_slice(&output.stdout)?;
    Ok(json)
}

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let stories = fetch_hn_top(10)?;
    for story in &stories {
        println!("{}", story["title"].as_str().unwrap_or(""));
    }
    Ok(())
}

rust

use std::process::Command;
use serde_json::Value;

fn fetch_hn_top(limit: u32) -> anyhow::Result<Vec<Value>> {
    let output = Command::new("opencli-rs")
        .args(["hackernews", "top", "--limit", &limit.to_string(), "--format", "json"])
        .output()?;

    let json: Vec<Value> = serde_json::from_slice(&output.stdout)?;
    Ok(json)
}

fn fetch_twitter_search(query: &str) -> anyhow::Result<Vec<Value>> {
    let output = Command::new("opencli-rs")
        .args(["twitter", "search", query, "--format", "json"])
        .output()?;

    if !output.status.success() {
        let err = String::from_utf8_lossy(&output.stderr);
        anyhow::bail!("opencli-rs error: {}", err);
    }

    let json: Vec<Value> = serde_json::from_slice(&output.stdout)?;
    Ok(json)
}

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let stories = fetch_hn_top(10)?;
    for story in &stories {
        println!("{}", story["title"].as_str().unwrap_or(""));
    }
    Ok(())
}

AI Agent integration (AGENT.md / .cursorrules)

AI Agent集成（AGENT.md / .cursorrules）

markdown

undefined

markdown

undefined

AGENT.md

Available Tools

可用工具

Run

opencli-rs list

to discover all available commands.

运行

opencli-rs list

查看所有可用命令。

Web Data Fetching

网页数据获取

```
opencli-rs hackernews top --format json
```
— HN top stories

opencli-rs twitter search "<query>" --format json

— Twitter search

opencli-rs arxiv search "<topic>" --format json

— Research papers

opencli-rs reddit subreddit <name> --format json

— Subreddit posts

```
opencli-rs hackernews top --format json
```
— HN热门文章

opencli-rs twitter search "<query>" --format json

— Twitter搜索

opencli-rs arxiv search "<topic>" --format json

— 学术论文

opencli-rs reddit subreddit <name> --format json

— 子版块帖子

Local CLI Tools

本地CLI工具

```
opencli-rs gh <args>
```
— GitHub operations
```
opencli-rs docker <args>
```
— Docker operations
```
opencli-rs kubectl <args>
```
— Kubernetes operations

Register custom tools:

opencli-rs register mycli

---

```
opencli-rs gh <args>
```
— GitHub操作
```
opencli-rs docker <args>
```
— Docker操作
```
opencli-rs kubectl <args>
```
— Kubernetes操作

注册自定义工具：

opencli-rs register mycli

---

Configuration

配置

Config file location

配置文件位置

~/.config/opencli-rs/config.toml   # macOS / Linux
%APPDATA%\opencli-rs\config.toml   # Windows

~/.config/opencli-rs/config.toml   # macOS / Linux
%APPDATA%\opencli-rs\config.toml   # Windows

Custom adapter directory

自定义适配器目录

toml

undefined

toml

undefined

~/.config/opencli-rs/config.toml

adapter_dir = "~/.config/opencli-rs/adapters" default_format = "table" default_limit = 20

undefined

adapter_dir = "~/.config/opencli-rs/adapters" default_format = "table" default_limit = 20

undefined

Register a custom external CLI

注册自定义外部CLI

bash

undefined

bash

undefined

Register your own CLI tool for passthrough

注册自己的CLI工具以支持透传调用

opencli-rs register mycli

Now use it via opencli-rs

现在可通过opencli-rs调用

opencli-rs mycli --help opencli-rs mycli some-command --flag value

---

opencli-rs mycli --help opencli-rs mycli some-command --flag value

---

Troubleshooting

故障排查

Run diagnostics first

先运行诊断

bash

opencli-rs doctor

bash

opencli-rs doctor

Chrome extension not connecting

Chrome扩展无法连接

bash

undefined

bash

undefined

Check extension is loaded at chrome://extensions

检查扩展是否已在chrome://extensions加载

Verify Developer mode is ON

确认开发者模式已开启

Reload the extension after reinstalling opencli-rs

重新安装opencli-rs后重启扩展

Check daemon is running:

检查守护进程是否运行：

opencli-rs doctor

undefined

opencli-rs doctor

undefined

Browser command returns empty / auth error

浏览器命令返回空结果/认证错误

Open Chrome and ensure you are logged in to the target site
The extension reuses your existing browser session — no tokens needed
Try refreshing the target site tab, then retry the command

打开Chrome并确保已登录目标站点
扩展会复用你现有的浏览器会话 — 无需额外令牌
尝试刷新目标站点标签页后重试命令

Binary not found after install

安装后找不到二进制文件

bash

undefined

bash

undefined

Verify install location

验证安装位置

which opencli-rs ls /usr/local/bin/opencli-rs

Add to PATH if missing

若未在PATH中，添加路径

export PATH="/usr/local/bin:$PATH" echo 'export PATH="/usr/local/bin:$PATH"' >> ~/.zshrc

undefined

export PATH="/usr/local/bin:$PATH" echo 'export PATH="/usr/local/bin:$PATH"' >> ~/.zshrc

undefined

Build errors from source

源码构建错误

bash

undefined

bash

undefined

Ensure Rust toolchain is up to date

确保Rust工具链是最新版本

rustup update stable rustup target add aarch64-apple-darwin # macOS Apple Silicon cargo build --release

undefined

rustup update stable rustup target add aarch64-apple-darwin # macOS Apple Silicon cargo build --release

undefined

Command slow or timing out

命令运行缓慢或超时

Browser commands require Chrome + extension; without it they will hang
Run
```
opencli-rs doctor
```
to verify extension connectivity
Public commands (hackernews, arxiv, etc.) need no browser and run in ~1-2s

浏览器模式需要Chrome及扩展；若无则会挂起
运行
```
opencli-rs doctor
```
验证扩展连接性
公共模式命令（hackernews、arxiv等）无需浏览器，运行时间约1-2秒

Windows path issues

Windows路径问题

powershell

undefined

powershell

undefined

Verify binary location

验证二进制文件位置

Get-Command opencli-rs

If not found, ensure the directory is in $env:PATH

若未找到，确保目录在$env:PATH中

$env:PATH += ";$env:LOCALAPPDATA\Microsoft\WindowsApps"

---

$env:PATH += ";$env:LOCALAPPDATA\Microsoft\WindowsApps"

---

Supported Sites Reference

支持站点参考

Category	Sites
Tech News	HackerNews, Dev.to, Lobsters, Linux-do
Social	Twitter/X, Reddit, Facebook, Instagram, TikTok, Jike
Video	YouTube, Bilibili, Weixin
Chinese	Zhihu, Xiaohongshu, Weibo, Douban, Xueqiu, Weread, Sinablog, Sinafinance
Research	arXiv, Hugging Face
Finance	Yahoo Finance, Barchart, Xueqiu
Jobs	Boss, LinkedIn
Reading	Medium, Substack, Wikipedia, BBC, Bloomberg, Reuters
Shopping	Steam, SMZDM, Ctrip, Coupang
AI/Desktop	Cursor, ChatGPT, Codex, Doubao, ChatWise, Notion, Discord
Podcast	Apple Podcasts, Xiaoyuzhou
External CLI	gh, docker, kubectl, obsidian, readwise, gws

分类	站点
科技资讯	HackerNews, Dev.to, Lobsters, Linux-do
社交平台	Twitter/X, Reddit, Facebook, Instagram, TikTok, 即刻
视频平台	YouTube, B站, 微信视频号
中文平台	知乎, 小红书, 微博, 豆瓣, 雪球, 微信读书, 新浪博客, 新浪财经
学术研究	arXiv, Hugging Face
金融财经	Yahoo Finance, Barchart, 雪球
招聘求职	BOSS直聘, LinkedIn
阅读平台	Medium, Substack, Wikipedia, BBC, Bloomberg, Reuters
购物平台	Steam, 什么值得买, 携程, Coupang
AI/桌面应用	Cursor, ChatGPT, Codex, 豆包, ChatWise, Notion, Discord
播客平台	Apple Podcasts, 小宇宙
外部CLI工具	gh, docker, kubectl, obsidian, readwise, gws

opencli-rs-web-scraper

Original

Translation

opencli-rs Web Scraper & Site Fetcher

opencli-rs 网页抓取与站点信息获取工具

Installation

安装

One-line (macOS / Linux)

一键安装（macOS / Linux）

Windows (PowerShell)

Windows（PowerShell）

Build from Source

从源码构建

Chrome Extension (required for Browser-mode commands)

Chrome扩展（浏览器模式命令必需）

AI Agent Skill Install

AI Agent Skill安装

Command Modes

命令模式

Key Commands

核心命令

Global Flags

全局参数

Public Commands (no browser needed)

公共模式命令（无需浏览器）

Hacker News

Hacker News

arXiv

arXiv

Wikipedia

Wikipedia

Stack Overflow

Stack Overflow

Dev.to

Dev.to

Linux.do

Linux.do

Browser Commands (Chrome extension required)

浏览器模式命令（需Chrome扩展）

Twitter/X

Twitter/X

Bilibili

B站

Reddit

Reddit

Zhihu

知乎

Xiaohongshu

小红书

YouTube

YouTube

Weibo

微博

Douban

豆瓣

Medium

Medium

Xueqiu (stock)

雪球（股票）

Desktop App Control (Electron)

桌面应用控制（Electron）

Cursor IDE

Cursor IDE

ChatGPT Desktop

ChatGPT桌面端

Notion

Notion

Discord Desktop

Discord桌面端

External CLI Passthrough

外部CLI透传调用

GitHub CLI

GitHub CLI

Docker

Docker

Kubernetes

Kubernetes

AI Discovery Commands

AI发现命令

Explore a website's API surface