crawl-xueqiu-user-timeline

Original：🇨🇳 Chinese
Translated
3 scripts

Crawl Xueqiu user's post timeline and save as Markdown file

2installs
Sourcecnife/my-scripts
Added on

NPX Install

npx skill4agent add cnife/my-scripts crawl-xueqiu-user-timeline

Tags

Translated version includes tags in frontmatter
xueqiu-scrapingdata-crawlingmarkdown-exportbrowser-automationtimeline-extraction

SKILL.md Content (Chinese)

View Translation Comparison →

crawl-xueqiu-user-timeline

Crawl Xueqiu user's post timeline and save as Markdown file.

Prerequisites

Base directory for this skill: 
{base_dir}
Replace 
{base_dir}
with the actual path when calling, for example: 
/home/cnife/code/try-agent-browser-automation/.agents/skills/crawl-xueqiu-user-timeline
Ensure Chrome is in Debug mode and agent-browser is installed:
bash
sh {base_dir}/scripts/check-cdp.sh
sh {base_dir}/scripts/check-agent-browser.sh

Usage

Run the crawling script directly:
bash
{base_dir}/scripts/crawl_xueqiu_user_timeline_api.py <Xueqiu user homepage URL> [options]

Parameter Description

ParameterDescriptionDefault Value
url
Xueqiu user homepage URLRequired
--days
Crawl posts from the last N days3
--start-date
Start date (YYYY-MM-DD)3 days ago
--end-date
End date (YYYY-MM-DD)Today
-o, --output
Output file nameAuto-generated
Note: Parameters 
--days
and 
--start-date
are mutually exclusive and cannot be used together.

Examples

bash
# Crawl the last 3 days (default)
{base_dir}/scripts/crawl_xueqiu_user_timeline_api.py https://xueqiu.com/u/9493911686

# Crawl the last 7 days
{base_dir}/scripts/crawl_xueqiu_user_timeline_api.py https://xueqiu.com/u/9493911686 --days 7

# Crawl the last 30 days
{base_dir}/scripts/crawl_xueqiu_user_timeline_api.py https://xueqiu.com/u/9493911686 --days 30

# Specify date range
{base_dir}/scripts/crawl_xueqiu_user_timeline_api.py https://xueqiu.com/u/9493911686 --start-date 2026-01-01 --end-date 2026-03-05

# Specify output file name
{base_dir}/scripts/crawl_xueqiu_user_timeline_api.py https://xueqiu.com/u/9493911686 -o my_timeline.md

Output Format

The generated Markdown file includes:
  • Basic user information (UID, followers, followings, post count)
  • Chronologically sorted post records
  • Each post includes: publication time, content, quoted content (if any), interaction data, original post link

Notes

  1. You need to log in to your Xueqiu account first
  2. If you encounter the Verification page, complete the verification manually and run the script again
  3. Pagination and md5__1038 token are automatically handled during crawling
  4. The output file is saved in the current working directory
Need further analysis?
After crawling is completed, ask the user: Do you need to summarize and analyze the Xueqiu user's posts?