Loading...
Found 1 Skills
Discover all URLs on a website without fetching content — fast, low-cost URL inventory via sitemap.xml + link extraction BFS. Use when you need to know which pages exist before deciding what to scrape or crawl: "list all pages", "find URLs on this site", "discover links", "what pages does this site have", "map the site". Step 3 of the crw workflow ladder.