Competitive Intelligence

Prerequisites

• Apify CLI v1.5.0+ (

  npm install -g apify-cli

  ), or Apify MCP server
• Authenticated session (

  apify login

  or

  APIFY_TOKEN

  env var)

--json

--user-agent apify-awesome-skills/apify-easy-competitive-intelligence

2>/dev/null

• Run actor:

  apify actors call "ACTOR_ID" -i 'INPUT' --user-agent apify-awesome-skills/apify-easy-competitive-intelligence --json 2>/dev/null

  → returns run metadata with

  defaultDatasetId

• Fetch results:

  apify datasets get-items DATASET_ID --user-agent apify-awesome-skills/apify-easy-competitive-intelligence --format json > /tmp/results.json 2>/dev/null

  — save locally, parse from file:
  • Quick extraction:

    jq '.[] | "\(.field1) | \(.field2)"' /tmp/results.json

  • Aggregation:

    python3 -c "import json; d=json.load(open('/tmp/results.json')); ..."

  • Tabular:

    --format csv > /tmp/results.csv

    +

    python3

    with

    csv.DictReader

  • Flags:

    --limit N

    ,

    --offset N

    ,

    --format json|jsonl|csv|xlsx|xml

  • Output fields:

    apify datasets info DATASET_ID --json | jq .fields

• Fetch schema:

  apify actors info "ACTOR_ID" --input --user-agent apify-awesome-skills/apify-easy-competitive-intelligence --json 2>/dev/null

call-actor

fetch-actor-details

get-actor-output

Authentication

1. OAuth (interactive):

   apify login

   (opens browser)
2. Environment variable:

   export APIFY_TOKEN=your_token_here

3. From .env file:

   source .env

   (if the file contains

   APIFY_TOKEN=...

   )

Actor Registry

1. Read — find the actor's section in

   reference/actor-schemas.md

   . Use the exact verified input and follow the "How to find" instructions for URLs/slugs.
2. Discover — verify platform URLs and slugs (e.g. via SERP) as described in the actor's schema section. Do not guess — wrong slugs silently return empty or wrong data.
3. Run — call the actor with verified input.

apify actors info "ACTOR_ID" --user-agent apify-awesome-skills/apify-easy-competitive-intelligence --input --json 2>/dev/null

apify/google-search-scraper

apify/website-content-crawler

apify/rag-web-browser

dev_fusion/Linkedin-Company-Scraper

curious_coder/linkedin-jobs-scraper

pratikdani/crunchbase-companies-scraper

junglee/Amazon-crawler

web_wanderer/amazon-reviews-extractor

e-commerce/walmart-product-detail-scraper

compass/Google-Maps-Reviews-Scraper

automation-lab/g2-scraper

zen-studio/capterra-reviews-scraper

memo23/glassdoor-scraper-ppr

harshmaur/reddit-scraper

neatrat/google-play-store-reviews-scraper

jdtpnjtp/apple-app-store-scraper

pro100chok/similarweb-scraper

data_xplorer/google-news-scraper-fast

andok/wayback-machine-scraper

Core Workflow

Step 0: Understand the User (once, at start)

• Role — Analyzed company, competitor, investor, consultant?
• Decision — Entering market, defending position, choosing vendor, building battlecard?
• Autonomy — Checkpoints after initial findings, or autopilot?

Steps 1–7

1. Clarify scope — Identify competitors. Select module(s). Default geography: US.
2. Read module reference — Load

   reference/modules/<module>.md

   for gathering + analysis instructions.
3. Gather live data — For each actor call, follow the three-step pattern: Read (actor-schemas.md) → Discover (SERP for URLs) → Run (call actor). Use PRIMARILY actors from the Actor Registry above.
4. Checkpoint (if not autopilot) — Present first findings, confirm direction.
5. Analyze — Select framework, lead with narrative, support with tables.
6. Verify — Run pre-delivery verification (

   reference/verification-checklist.md

   ). Check: every claim has a source URL, every major finding has a confidence label, inferences are labeled as such. Remove any ungrounded claims.
7. Deliver — End with strategic recommendations framed for the user's role.

Framework Selection

Data Collection Rules

• Prefer structured actors over

  website-content-crawler

  when a dedicated actor exists.
• Cost budget — 3-8 actor calls per snapshot. Track total, warn at 15+.
• Parallelize independent

  call-actor

  calls in a single response.
• Failures — Report every failure explicitly (actor, input, error). Retry with corrected input if the cause is obvious. If retry fails, try

  rag-web-browser

  as fallback. Never silently skip a failed data source.
• Cite everything — Include source URLs for every data point.
• Async for long runs — Set

  async: true

  for actors >30s, poll with

  get-actor-run

  .
• Protected platforms — Do NOT use

  website-content-crawler

  or

  rag-web-browser

  for: g2.com, capterra.com, gartner.com, glassdoor.com, reddit.com, linkedin.com. Use dedicated actors.

Apify vs. WebSearch

Data Validation & Grounding

• Every factual claim needs a source URL. No link = not a fact.
• Confidence labels are mandatory. Mark every major finding: High (primary source), Medium (2+ third-party sources), Low (single third-party source). Format:

  [Confidence | Source]

  . No report without labels.
• Data tiers: Verified (primary source) → Reported (third-party, attribute) → Inferred (label as "this suggests...") → Ungrounded (omit).
• Numbers are dangerous — employee counts, revenue, funding change fast. Always cite source and date.
• Empty results ARE intelligence — 0 jobs = not hiring, 0 SimilarWeb = small site, 12 reviews = low adoption.
• Cross-reference — Single-source claims are unverified. Multi-source (G2 + Capterra + Reddit) = pattern.

Module Selection

reference/modules/competitor-snapshot.md

reference/modules/pricing-intelligence.md

reference/modules/pricing-deep-dive.md

reference/modules/review-intelligence.md

reference/modules/hiring-signals.md

reference/modules/content-seo.md

reference/modules/market-landscape.md

reference/multi-module-playbook.md