Loading...
Loading...
Web crawling and scraping tool with LLM-optimized output. 网页爬虫爬取工具 | Web crawler, web scraper, spider. DuckDuckGo search, site crawling, dynamic page scraping. 智能搜索爬取 | Free, no API key required.
npx skill4agent add lancelin111/crawl4ai-skill crawl4ai-skillpip install crawl4ai-skill# Search the web with DuckDuckGo
crawl4ai-skill search "python web scraping"# Scrape a single web page
crawl4ai-skill crawl https://example.com# Crawl entire website / spider
crawl4ai-skill crawl-site https://docs.python.org --max-pages 50# Crawl documentation site with spider
crawl4ai-skill crawl-site https://docs.fastapi.com --max-pages 100# Search and scrape top results
crawl4ai-skill search-and-crawl "Vue 3 best practices" --crawl-top 3# Scrape JavaScript-heavy pages
crawl4ai-skill crawl https://xueqiu.com/S/BIDU --wait-until networkidle --delay 2| 命令 Command | 说明 Description |
|---|---|
| Web search 网页搜索 |
| Web scraping 单页爬取 |
| Web crawling 全站爬虫 |
| Search + scrape 搜索并爬取 |
# Web Search 搜索
--num-results 10 # Number of results
# Web Scraping 爬取
--format fit_markdown # Output format
--output result.md # Output file
--wait-until networkidle # Wait strategy for dynamic pages
--delay 2 # Additional wait time (seconds)
--wait-for ".selector" # Wait for specific element
# Web Crawling 爬虫
--max-pages 100 # Max pages to crawl
--max-depth 3 # Max crawl depthcrawl4ai-skill crawl https://example.com --format fit_markdowncrawl4ai-skill crawl https://example.com --format raw_markdown