xfetch

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

xfetch

Fetch public data from X (Twitter) — profiles, tweets, threads, search, followers — using the user's own login cookies instead of the paid developer API. This skill ships a single self-contained CLI,

scripts/xfetch.py

, that calls X's GraphQL API directly and emits structured data an agent can parse.

How it stays working (the architecture that matters): the hard part of X scraping is that X constantly changes its site — it rotates GraphQL query IDs every few weeks, migrates endpoints between GET and POST, and reshapes response JSON. This CLI splits those concerns so a break in one place doesn't take everything down:

It reuses the current query IDs and feature flags shipped by the twikit package (kept fresh with
```
pip install -U twikit
```
) — so you don't hand-maintain the fastest-rotating values.
It does its own HTTP requests and JSON parsing, which sidesteps the two things most likely to be broken in twikit at any moment: (1) its homepage-scraped anti-bot transaction ID — this CLI sends a harmless placeholder
```
x-client-transaction-id
```
, which X accepts for read endpoints; and (2) its response-model layer — this CLI parses the raw JSON itself. It also auto-falls-back GET↔POST on a 404, so an endpoint X migrates keeps working.

Net: if a command suddenly errors for everyone, the first fix is still

pip install -U twikit

(refreshes query IDs). If that doesn't help, X changed a response shape — check the parser in

scripts/xfetch.py

使用用户自己的登录Cookie而非付费开发者API，从X（Twitter）获取公开数据——包括资料、推文、推文线程、搜索结果、粉丝列表等。本技能提供一个独立的CLI工具

scripts/xfetch.py

，它直接调用X的GraphQL API，并输出可供Agent解析的结构化数据。

持续可用的核心设计（关键架构）： X抓取的难点在于X的网站一直在变化——每隔几周就会轮换GraphQL查询ID，在GET和POST之间切换端点，还会调整响应JSON的结构。这个CLI将这些关注点分离，因此某一出问题不会导致整个工具失效：

它复用twikit包提供的当前查询ID和功能标志（通过
```
pip install -U twikit
```
保持更新）——这样你就不用手动维护那些更新最频繁的值。
它自行处理HTTP请求和JSON解析，这避开了twikit随时可能出问题的两个地方：(1) 它从首页抓取的反机器人事务ID——这个CLI发送一个无害的占位符
```
x-client-transaction-id
```
，X的读取端点会接受这个值；(2) 它的响应模型层——这个CLI自己解析原始JSON。它还会在遇到404时自动在GET和POST之间切换，因此X迁移的端点仍能正常工作。

总而言之，如果某个命令突然对所有人都报错，第一个修复方法仍然是

pip install -U twikit

（更新查询ID）。如果这没用，那就是X修改了响应结构——检查

scripts/xfetch.py

中的解析器。

The agent-native contract

Agent原生约定

scripts/xfetch.py

is built so you can drive it in a loop without surprises. Rely on these three channels rather than scraping prose:

stdout
carries the data payload only (JSON by default). Redirect or pipe it — nothing else is written there.
stderr
carries human status (
```
✓ Saved…
```
) and errors. Read it to diagnose; ignore it when parsing.
exit code is the truth signal:
```
0
```
= success,
```
1
```
= error,
```
2
```
= not authenticated,
```
3
```
= rate limited (stderr carries
```
rate_limit_reset=<epoch>
```
),
```
64
```
= usage error (bad flag or argument — fix the command, don't re-auth). Branch on the exit code, not on stderr text.

scripts/xfetch.py

的设计让你可以循环调用它而不会出意外。请依赖以下三个通道，而非解析文本输出：

stdout
仅承载数据 payload（默认是JSON）。可以重定向或通过管道传输它——那里不会输出其他内容。
stderr
承载人类可读的状态（
```
✓ 已保存…
```
）和错误。用它来诊断问题；解析数据时忽略它。
退出码是可信信号：
```
0
```
= 成功，
```
1
```
= 错误，
```
2
```
= 未认证，
```
3
```
= 速率限制（stderr中会包含
```
rate_limit_reset=<epoch>
```
），
```
64
```
= 使用错误（标志或参数错误——修正命令即可，不要重新认证）。请根据退出码分支处理，不要依据stderr文本判断。

Setup (once per environment)

安装设置（每个环境只需一次）

Confirm Python and install twikit (a virtualenv is cleanest so you don't touch system packages):
bash
```
python3 -m pip install -r scripts/requirements.txt   # installs twikit>=2.3.3
```

Verify the CLI loads:

bash

python3 scripts/xfetch.py --version    # -> xfetch-skill 0.3

Invoke every command as

python3 scripts/xfetch.py <command> …

using this skill's own copy of the script.

确认Python环境并安装twikit（使用虚拟环境最干净，不会影响系统包）：
bash
```
python3 -m pip install -r scripts/requirements.txt   # installs twikit>=2.3.3
```

验证CLI可正常加载：

bash

python3 scripts/xfetch.py --version    # -> xfetch-skill 0.3

所有命令都通过

python3 scripts/xfetch.py <命令> …

的方式调用，使用本技能自带的脚本副本。

Authenticate first — this is the #1 failure mode

首先完成认证——这是最常见的失败原因

Every data command needs the user's X cookies (

auth_token

ct0

). Without them the command prints

Not authenticated.

to stderr and exits 2. Auth is delegated: the human sets it up once, the CLI saves a session to

~/.config/xfetch/cookies.json

, and you reuse it — you don't own the credential lifecycle, you just check it's present.

Always run this first and branch on the exit code:

bash

python3 scripts/xfetch.py auth check

If it isn't authenticated, get the human to provide credentials, in order of preference:

Extract from a logged-in browser (most convenient — reads the x.com cookies straight from the browser's store, no copy/paste):
bash
```
python3 scripts/xfetch.py auth extract --browser chrome
# supported: chrome, chromium, firefox, safari, edge, brave, arc, opera, vivaldi, librewolf
```
Requires
```
browser_cookie3
```
(in
```
requirements.txt
```
). On macOS this may prompt for Keychain access — that's the OS handing over the cookie-decryption key, which is expected.
Import a cookies file (best for browsers without a reader — e.g. ChatGPT Atlas, Orion, or any Chromium fork): export cookies with a "Get cookies.txt" browser extension (Netscape
```
cookies.txt
```
or JSON both work), then point the CLI at the file — it extracts
```
auth_token
```
+
```
ct0
```
for you:
bash
```
python3 scripts/xfetch.py auth import --file ~/Downloads/x.com_cookies.txt
```
Paste cookie tokens (works for any browser): open DevTools → Application → Cookies →
```
https://x.com
```
, copy the
```
auth_token
```
and
```
ct0
```
values, then:
bash
```
python3 scripts/xfetch.py auth set --auth-token <token> --ct0 <token>
```

Environment variables (good for CI / one-off, saves nothing to disk):

bash

XFETCH_AUTH_TOKEN=<token> XFETCH_CT0=<token> python3 scripts/xfetch.py user @handle

Username/password login (twikit performs a real login; may trip a captcha or 2FA — pass
```
--totp
```
for TOTP-based 2FA):
bash
```
python3 scripts/xfetch.py auth login --username <user> --password <pass>
```

Never invent or guess cookie values. Prefer

auth set

/env over

login

so no password passes through the command line. If auth is missing and you can't obtain it, stop and ask the human — you cannot proceed without it. These read a real account, so its login is a shared, rate-limited resource: don't hammer it.

所有数据命令都需要用户的X Cookie（

auth_token

ct0

）。没有它们的话，命令会向stderr打印

Not authenticated.

并以退出码2退出。认证是委托式的：用户只需设置一次，CLI会将会话保存到

~/.config/xfetch/cookies.json

，之后你就可以复用它——你不需要管理凭证的生命周期，只需要检查它是否存在。

务必先运行这个命令并根据退出码判断：

bash

python3 scripts/xfetch.py auth check

如果未认证，让用户提供凭证，优先选择以下方式：

从已登录的浏览器中提取（最方便——直接从浏览器存储中读取x.com的Cookie，无需复制粘贴）：
bash
```
python3 scripts/xfetch.py auth extract --browser chrome
# supported: chrome, chromium, firefox, safari, edge, brave, arc, opera, vivaldi, librewolf
```
需要
```
browser_cookie3
```
（已包含在
```
requirements.txt
```
中）。在macOS上这可能会请求钥匙串访问权限——这是系统在提供Cookie解密密钥，属于正常现象。
导入Cookie文件（最适合没有读取器的浏览器——例如ChatGPT Atlas、Orion，或任何Chromium衍生浏览器）：使用“Get cookies.txt”浏览器扩展导出Cookie（Netscape格式的
```
cookies.txt
```
或JSON都可以），然后让CLI指向该文件——它会自动为你提取
```
auth_token
```
+
```
ct0
```
：
bash
```
python3 scripts/xfetch.py auth import --file ~/Downloads/x.com_cookies.txt
```
粘贴Cookie令牌（适用于所有浏览器）：打开开发者工具 → 应用 → Cookie →
```
https://x.com
```
，复制
```
auth_token
```
和
```
ct0
```
的值，然后运行：
bash
```
python3 scripts/xfetch.py auth set --auth-token <token> --ct0 <token>
```

环境变量（适合CI/一次性使用，不会保存到磁盘）：

bash

XFETCH_AUTH_TOKEN=<token> XFETCH_CT0=<token> python3 scripts/xfetch.py user @handle

用户名/密码登录（twikit会执行真实登录；可能会触发验证码或2FA——如果是TOTP类型的2FA请传入
```
--totp
```
）：
bash
```
python3 scripts/xfetch.py auth login --username <user> --password <pass>
```

绝不要编造或猜测Cookie值。优先使用

auth set

或环境变量而非

login

，这样密码不会经过命令行。如果缺少认证信息且你无法获取，请停止并询问用户——没有它你无法继续操作。这些操作会读取真实账号，因此该账号的登录是共享的、有速率限制的资源：不要频繁发起请求。

Choose the output format for the consumer

为使用方选择合适的输出格式

The default

json

is pretty-printed for a human reading one result. For anything you or a pipeline will parse, pick deliberately with

--format

Format	Flag	Use when
JSON (pretty)	(default)	Inspecting a single object or a small result
JSONL	`--format jsonl`	Large / multi-page results — one record per line, appendable, stream-friendly
CSV	`--format csv`	Spreadsheets or quick tabular analysis (nested fields are JSON-encoded in-cell)
SQLite	`--format sqlite --db out.db`	Building a queryable dataset; rows go to a `tweets` / `users` table, a summary prints to stderr

Add

--plain

for compact single-line JSON. Save to a file by redirecting stdout, e.g.

… --format jsonl > tweets.jsonl

默认的

json

格式是美化打印的，适合人类查看单个结果。对于你或流水线要解析的内容，请使用

--format

主动选择：

格式	标志	适用场景
JSON（美化）	(默认)	检查单个对象或少量结果时
JSONL	`--format jsonl`	大量/多页结果——每行一条记录，可追加，适合流式处理
CSV	`--format csv`	电子表格或快速表格分析（嵌套字段会以JSON编码的形式放在单元格中）
SQLite	`--format sqlite --db out.db`	构建可查询的数据集；行数据会存入 `tweets` / `users` 表，摘要信息会打印到stderr

添加

--plain

可获得紧凑的单行JSON。通过重定向stdout保存到文件，例如

… --format jsonl > tweets.jsonl

。

Commands

命令

bash

undefined

bash

undefined

Profile (by @handle or numeric id)

python3 scripts/xfetch.py user @elonmusk

A user's tweets (--replies for replies, --media for media-only)

python3 scripts/xfetch.py tweets @elonmusk -n 50 python3 scripts/xfetch.py tweets @elonmusk --replies --all --format jsonl > timeline.jsonl

A user's liked tweets

python3 scripts/xfetch.py likes @handle -n 40

Single tweet, and a tweet with its replies (thread)

python3 scripts/xfetch.py tweet https://x.com/user/status/1234567890 python3 scripts/xfetch.py thread 1234567890

Search (--type top|latest|media)

python3 scripts/xfetch.py search "AI agents" -n 100 --type latest python3 scripts/xfetch.py search "from:openai since:2024-01-01" --all --format csv > openai.csv

Followers / following, paginated into a SQLite dataset

python3 scripts/xfetch.py followers @handle --all --format sqlite --db network.db python3 scripts/xfetch.py following @handle -n 100

Your own timelines (uses the logged-in account)

python3 scripts/xfetch.py home # "For You" python3 scripts/xfetch.py home --following # chronological python3 scripts/xfetch.py bookmarks -n 50


`search` supports X's advanced operators (`from:`, `to:`, `since:`, `until:`, `filter:`, `min_faves:`, etc.) — pass them inside the quoted query. For the full flag reference and the exact fields each command returns, read `references/commands.md`.

DMs, lists, and trends are not exposed as commands (to keep this CLI small), but twikit supports them (`get_dm_history`, `get_list_tweets`, `get_trends`) — add a command following the existing pattern in `scripts/xfetch.py` if the user needs one.

python3 scripts/xfetch.py home # "For You" python3 scripts/xfetch.py home --following # chronological python3 scripts/xfetch.py bookmarks -n 50


`search`支持X的高级运算符（`from:`、`to:`、`since:`、`until:`、`filter:`、`min_faves:`等）——将它们放在带引号的查询语句中即可。完整的参数参考和每个命令返回的具体字段，请查看`references/commands.md`。

私信、列表和趋势没有作为命令暴露（为了保持CLI轻量），但twikit支持这些功能（`get_dm_history`、`get_list_tweets`、`get_trends`）——如果用户需要，可以按照`scripts/xfetch.py`中现有的模式添加命令。

Pagination and rate limits

分页与速率限制

Listing commands (

tweets

search

followers

following

likes

home

bookmarks

) share these flags — single page by default, opt into more:

bash

-n 40            # results per page
--all            # every page until exhausted
--max-pages 10   # cap the pages
--cursor <c>     # start from a specific pagination cursor
--delay 1.5      # seconds between pages (default 1.0 — keep it >0 for --all)

When more pages remain — or a pull is interrupted or rate limited mid-way — the CLI prints

next_cursor=<value>

to stderr. Capture it and pass it back via

--cursor

to resume exactly where the pull stopped instead of refetching from the top.

X enforces per-account rate limits (roughly a few hundred requests per 15-minute window per endpoint) and this uses the user's real account. For bulk pulls keep a real

--delay

, prefer

--max-pages

over

--all

when you only need a sample, and remember aggressive scraping can get the account throttled or flagged. Route through a proxy for heavier work with

--proxy http://user:pass@host:port

列表类命令（

tweets

、

search

、

followers

、

following

、

likes

、

home

、

bookmarks

）共享以下参数——默认只获取单页，可选择获取更多：

bash

-n 40            # results per page
--all            # every page until exhausted
--max-pages 10   # cap the pages
--cursor <c>     # start from a specific pagination cursor
--delay 1.5      # seconds between pages (default 1.0 — keep it >0 for --all)

当还有更多页面时——或者拉取过程被中断或遇到速率限制时——CLI会向stderr打印

next_cursor=<value>

。捕获这个值并通过

--cursor

传入，就可以从上次停止的地方继续，而不用从头重新获取。

X对每个账号都有速率限制（每个端点大概每15分钟几百次请求），而这个工具使用的是用户的真实账号。对于批量拉取，请设置合理的

--delay

，如果只需要样本，优先使用

--max-pages

而非

--all

，请记住过度抓取可能会导致账号被限流或标记。对于更大量的工作，可以通过

--proxy http://user:pass@host:port

使用代理。

When something fails

出现问题时

Check the exit code, then read stderr to classify:

exit 2 /
Not authenticated
→ run the auth flow above.
exit 64 / usage error → the command line itself is wrong (unknown flag, bad value). Fix the invocation — do not re-run auth.
exit 3 / rate limited → stderr carries
```
rate_limit_reset=<epoch>
```
; wait until then, raise
```
--delay
```
, and resume from the
```
next_cursor=
```
value printed on stderr.
exit 1 with a GraphQL / parsing error that started suddenly for everyone → X likely changed its site. Upgrade twikit (
```
pip install -U twikit
```
) and retry.
Timeout / network errors mid-pull → resume from the
```
next_cursor=
```
line on stderr instead of refetching from the top.
Empty result → the account may be private (you only see what the logged-in account can), suspended, or the handle is wrong.

先检查退出码，再读取stderr进行分类：

退出码2 /
Not authenticated
→ 运行上述认证流程。
退出码64 / 使用错误 → 命令行本身有问题（未知标志、值错误）。修正调用方式——不要重新运行认证。
退出码3 / 速率限制 → stderr中会包含
```
rate_limit_reset=<epoch>
```
；等到那个时间，调高
```
--delay
```
，然后从stderr上打印的
```
next_cursor=
```
值处恢复。
退出码1，且突然对所有人都出现GraphQL/解析错误 → 很可能是X修改了网站。升级twikit（
```
pip install -U twikit
```
）后重试。
拉取过程中出现超时/网络错误 → 从stderr上的
```
next_cursor=
```
行处恢复，而不是从头重新获取。
结果为空 → 该账号可能是私密的（你只能看到登录账号可见的内容）、被暂停，或者用户名有误。

Scope and good-citizen notes

使用范围与合规注意事项

This reads public data plus whatever the logged-in account can see (its own home, bookmarks, likes). It does not post, like, follow, or modify anything — it's read-only by design. Because it uses the user's own session, the user is responsible for staying within X's Terms of Service and rate limits. Use it for legitimate purposes — research, personal archiving, monitoring accounts the user is entitled to read — not for harassment, mass surveillance, or evading a block. Session cookies live under

~/.config/xfetch/

; clear them with

auth clear

本工具读取公开数据以及登录账号可见的内容（自己的首页、书签、点赞）。它不会发布、点赞、关注或修改任何内容——设计上就是只读的。由于它使用用户自己的会话，用户有责任遵守X的服务条款和速率限制。请将其用于合法目的——研究、个人归档、监控你有权读取的账号——不要用于骚扰、大规模监控或绕过封禁。会话Cookie存储在

~/.config/xfetch/

下；可以用

auth clear

清除它们。