• When the user mentions "scraper" or related topics
• When the user mentions "scraping" or related topics
• When the user mentions "extrair dados web" or related topics
• When the user mentions "web scraping" or related topics
• When the user mentions "raspar dados" or related topics
• When the user mentions "coletar dados site" or related topics

• The task is unrelated to web scraper
• A simpler, more specific tool can handle the request
• The user needs general-purpose assistance without domain expertise

• Multi-strategy: WebFetch (static), Browser automation (JS-rendered), Bash/curl (APIs), WebSearch (discovery)
• Extraction modes: table, list, article, product, contact, FAQ, pricing, events, jobs, custom
• Output formats: Markdown tables (default), JSON, CSV
• Pagination: auto-detect and follow (page numbers, infinite scroll, load-more)
• Multi-URL: extract same structure across sources with comparison and diff
• Validation: confidence ratings (HIGH/MEDIUM/LOW) on every extraction
• Auto-escalation: WebFetch fails silently -> automatic Browser fallback
• Data transforms: cleaning, normalization, deduplication, enrichment
• Differential mode: detect changes between scraping runs

• If user provides a URL and clear data target, proceed directly to Phase 2. Do NOT ask unnecessary questions.
• If request is ambiguous (e.g. "scrape this site"), ask ONLY: "What specific data do you want me to extract from this page?"
• Default to Markdown table output. Mention alternatives only if relevant.
• Accept requests in any language. Always respond in the user's language.
• If user says "everything" or "all data", perform recon first, then present what's available and let user choose.

1. Use WebSearch to find the most relevant pages
2. Present top 3-5 URLs with descriptions
3. Let user choose which to scrape, or scrape all
4. Proceed to Phase 2 with selected URL(s)

1. Get tab context:

   tabs_context_mcp(createIfEmpty=true)

   -> get tabId
2. Navigate to URL:

   navigate(url=TARGET_URL, tabId=TAB)

3. Wait for content to load:

   computer(action="wait", duration=3, tabId=TAB)

4. Check for cookie/consent banners:

   find(query="cookie consent or accept button", tabId=TAB)

   • If found, dismiss it (prefer privacy-preserving option)
5. Read page structure:

   read_page(tabId=TAB)

   or

   get_page_text(tabId=TAB)

6. Locate target elements:

   find(query="[DESCRIPTION]", tabId=TAB)

7. Extract with JavaScript for precise data via

   javascript_tool

// Table extraction
const rows = document.querySelectorAll('TABLE_SELECTOR tr');
const data = Array.from(rows).map(row => {
  const cells = row.querySelectorAll('td, th');
  return Array.from(cells).map(c => c.textContent.trim());
});
JSON.stringify(data);

// List/card extraction
const items = document.querySelectorAll('ITEM_SELECTOR');
const data = Array.from(items).map(item => ({
  field1: item.querySelector('FIELD1_SELECTOR')?.textContent?.trim() || null,
  field2: item.querySelector('FIELD2_SELECTOR')?.textContent?.trim() || null,
  link: item.querySelector('a')?.href || null,
}));
JSON.stringify(data);

1. For lazy-loaded content, scroll and re-extract:

   computer(action="scroll", scroll_direction="down", tabId=TAB)

   then

   computer(action="wait", duration=2, tabId=TAB)

1. WebSearch to discover relevant URLs
2. WebFetch for initial content assessment
3. Browser automation for JS-heavy sections
4. Bash for post-processing (jq, python for data cleaning)

1. Use Browser

   javascript_tool

   to extract structured data:

const scripts = document.querySelectorAll('script[type="application/ld+json"]');
const data = Array.from(scripts).map(s => {
  try { return JSON.parse(s.textContent); } catch { return null; }
}).filter(Boolean);
JSON.stringify(data);

1. This often provides cleaner, more reliable data than DOM scraping
2. Fall back to DOM extraction only for fields not in structured data

1. Extract data from current page
2. Identify URL pattern (e.g.

   ?page=N

   ,

   /page/N

   ,

   &offset=N

   )
3. Iterate through pages up to user's max (default: 5 pages)
4. Show progress: "Extracting page 2/5..."
5. Concatenate all results, deduplicate if needed

1. Extract currently visible data
2. Record item count
3. Scroll down:

   computer(action="scroll", scroll_direction="down", tabId=TAB)

4. Wait:

   computer(action="wait", duration=2, tabId=TAB)

5. Extract newly loaded data
6. Compare count - if no new items after 2 scrolls, stop
7. Repeat until no new content or max iterations (default: 5)

1. Extract currently visible data
2. Find button:

   find(query="load more button", tabId=TAB)

3. Click it:

   computer(action="left_click", ref=REF, tabId=TAB)

4. Wait and extract new content
5. Repeat until button disappears or max iterations reached

• Use Browser automation with

  javascript_tool

  and user's CSS selectors
• Or use WebFetch with a prompt built from user's field descriptions
• Always confirm extracted schema with user before proceeding to multi-URL

1. Extract from the first URL to establish the data schema
2. Show user the first results and confirm the schema is correct
3. Extract from remaining URLs using the same schema
4. Add a

   source

   column/field to every record with the origin URL
5. Combine all results into a single output
6. Show progress: "Extracting 3/7 URLs..."

1. Exact match: rows with identical values in all fields -> keep first
2. Near match: rows with same key fields (name+source) but different details -> keep most complete (fewer nulls), flag in notes
3. Report: "Removed N duplicate rows" in delivery notes

Confidence: HIGH - 47 items extracted, all 6 fields populated, matches expected count from page analysis.

• [Any gaps, issues, or observations]
• [Transforms applied: deduplication, normalization, etc.]
• [Pages scraped if paginated: "Pages 1-5 of 12"]
• [Auto-escalation if it occurred: "Escalated from WebFetch to Browser"]

undefined

• Left-align text columns (

  :---

  ), right-align numbers (

  ---:

  )
• Consistent column widths for readability
• Include summary row for numeric data when useful (totals, averages)
• Maximum 10 columns per table; split wider data into multiple tables or suggest JSON format
• Truncate long cell values to 60 chars with

  ...

  indicator
• Use

  N/A

  for missing values, never leave cells empty
• For multi-page results, show combined table (not per-page)

• Use camelCase for keys (e.g.

  productName

  ,

  unitPrice

  )
• Wrap in metadata envelope:
  json

  {
    "metadata": {
      "source": "URL",
      "title": "Page Title",
      "extractedAt": "ISO-8601",
      "itemCount": 47,
      "fieldCount": 6,
      "confidence": "HIGH",
      "strategy": "A",
      "transforms": ["deduplication", "priceNormalization"],
      "notes": []
    },
    "data": [ ... ]
  }

• Pretty-print with 2-space indentation
• Numbers as numbers (not strings), booleans as booleans
• null for missing values (not empty strings)

• First row is always headers
• Quote any field containing commas, quotes, or newlines
• UTF-8 encoding with BOM for Excel compatibility
• Use

  ,

  as delimiter (standard)
• Include metadata as comments:

  # Source: URL

• Markdown:

  .md

  extension
• JSON:

  .json

  extension
• CSV:

  .csv

  extension
• Confirm path before writing
• Report full file path and item count after saving

• Add

  Source

  as the first column/field
• Use short identifiers for sources (domain name or user label)
• Group by source or interleave based on user preference
• Highlight differences if user asks for comparison
• Include summary: "Best price: $X at store-b.com"

• Compare current extraction with previous run
• Mark new items with

  [NEW]

• Mark removed items with

  [REMOVED]

• Mark changed values with

  [WAS: old_value]

• Include summary: "Changes since last run: +5 new, -2 removed, 3 modified"

• Maximum 1 request per 2 seconds for sequential page fetches
• For multi-URL jobs, process sequentially with pauses
• If a site returns 429 (Too Many Requests), stop and report to user

• If a page blocks access (403, CAPTCHA, login wall), report to user
• Do NOT attempt to bypass bot detection, CAPTCHAs, or access controls
• Do NOT scrape behind authentication unless user explicitly provides access
• Respect robots.txt directives when known

• Do NOT reproduce large blocks of copyrighted article text
• For articles: extract factual data, statistics, and structured info; summarize narrative content
• Always include source attribution (http://example.com) in output

• Extract ONLY what the user explicitly requested
• Warn user before collecting potentially sensitive data at scale (emails, phone numbers, personal information)
• Do not store or transmit extracted data beyond what the user sees

1. Explain the specific reason (JS rendering, bot detection, login, etc.)
2. Suggest alternatives (different URL, API if available, manual approach)
3. Never retry aggressively or escalate access attempts

• Extraction patterns: references/extraction-patterns.md CSS selectors, JavaScript snippets, JSON-LD parsing, domain tips.
• Output templates: references/output-templates.md Markdown, JSON, CSV templates with complete examples.
• Data transforms: references/data-transforms.md Cleaning, normalization, deduplication, enrichment patterns.

• Provide clear, specific context about your project and requirements
• Review all suggestions before applying them to production code
• Combine with other complementary skills for comprehensive analysis

• Using this skill for tasks outside its domain expertise
• Applying recommendations without understanding your specific context
• Not providing enough project context for accurate analysis