X.com Post Scraper

Quick Start

cd .opencode/skills/x-scraper/scripts
python3 scraper.py <username> [count]

python3 scraper.py example_user 15

/tmp/x_{username}_posts.json

Prerequisites

1. Python 3.11+: Check with

   python3 --version

2. Playwright: Check with

   python3 -c "import playwright"

3. Cookie file: Check with

   ls /tmp/x_cookies_pw.json

Common Workflows

First-time setup

Daily scraping

python3 scraper.py <username> [count]

Custom cookie file

python3 scraper.py <username> [count] --cookie-file /path/to/cookies.json

Troubleshooting

Output Format

{
  "index": 1,
  "username": "example_user",
  "postId": "1234567890123456789",
  "publishTime": "2025-12-03T18:28:32.000Z",
  "postLink": "https://x.com/example_user/status/1234567890123456789",
  "textContent": "Post text content...",
  "views": "471K",
  "likes": "1.1K",
  "retweets": "153",
  "replies": "44"
}

• postLink

  - Direct URL to post

• publishTime

  - ISO 8601 timestamp

• views/likes/retweets/replies

  - Abbreviated metrics (K, M)

When to Use This Skill

• "整理 @某人 最近的发言"
• "看看某人在X上说了什么"
• "Scrape X.com posts from @username"
• "Get latest tweets from user"
• "Analyze X user's recent posts"

Available Scripts

scraper.py

- Main scraper

python3 scraper.py <username> [count] [--cookie-file <path>]

• Scrapes user timeline with replies
• Default count: 10 posts
• Default cookie:

  /tmp/x_cookies_pw.json

convert_cookies.py

- Cookie converter

python3 convert_cookies.py <input-file> [output-file]

• Converts Cookie-Editor JSON to Playwright format
• Required before first scraping

Reference Documents

• setup.md - Complete environment setup guide (Python, Playwright, cookies)
• troubleshooting.md - Error diagnosis and solutions
• usage.md - Detailed usage examples and advanced options

Limitations

• Requires X.com authentication cookies
• Cookies expire (~7 days), need re-export
• Rate limits may apply
• Cannot access private/protected accounts