scraping-google-search-results

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Scraping Google Search Results

抓取Google搜索结果

Raw SERP dataset for any search query. Returns organic results, featured snippets, People Also Ask boxes, and other SERP features.

针对任意搜索查询词的原始SERP数据集。返回自然搜索结果、精选摘要、“人们还问”模块以及其他SERP特征。

Prerequisites

前提条件

```
APIFY_TOKEN
```
environment variable set
Optional: Apify MCP server installed

已设置
```
APIFY_TOKEN
```
环境变量
可选：已安装Apify MCP服务器

Inputs

输入参数

Parameter	Type	Required	Default	Notes
`startUrls`	array	Optional	`[]`	Google search URLs
`searchTerms`	array	Optional	`[]`	Keywords to search on Google
`countryCode`	string	Optional	`US`	Country for Google search (e.g. `US` , `GB` , `TR` )
`languageCode`	string	Optional	—	Language for results (e.g. `en` )
`maxItems`	number	Optional	Unlimited	Maximum results to return across all queries
`maxPagesPerQuery`	integer	Optional	`1`	Maximum result pages per query
`mobileResults`	boolean	Optional	`false`	Fetch mobile SERP layout
`customMapFunction`	string	Optional	—	JavaScript function to transform each output object

参数	类型	是否必填	默认值	说明
`startUrls`	数组	可选	`[]`	Google搜索URL
`searchTerms`	数组	可选	`[]`	要在Google上搜索的关键词
`countryCode`	字符串	可选	`US`	Google搜索的国家/地区（例如 `US` 、 `GB` 、 `TR` ）
`languageCode`	字符串	可选	—	结果的语言（例如 `en` ）
`maxItems`	数字	可选	无限制	所有查询返回的最大结果数
`maxPagesPerQuery`	整数	可选	`1`	每个查询的最大结果页数
`mobileResults`	布尔值	可选	`false`	获取移动端SERP布局
`customMapFunction`	字符串	可选	—	用于转换每个输出对象的JavaScript函数

Workflow

工作流程

Progress:
- [ ] Step 1: Normalize query list
- [ ] Step 2: Run google-search-scraper
- [ ] Step 3: Poll for SUCCEEDED
- [ ] Step 4: Deliver SERP dataset

Progress:
- [ ] Step 1: Normalize query list
- [ ] Step 2: Run google-search-scraper
- [ ] Step 3: Poll for SUCCEEDED
- [ ] Step 4: Deliver SERP dataset

Step 2: Run the Actor

步骤2：运行Actor

Recommended — run_actor.js (handles waiting, output, and file saving automatically):

bash

undefined

推荐方式 — run_actor.js（自动处理等待、输出和文件保存）：

bash

undefined

Quick answer (prints table to chat)

node scripts/run_actor.js
--actor "apidojo~google-search-scraper"
--input '{"param": "value"}'

Save as CSV

node scripts/run_actor.js
--actor "apidojo~google-search-scraper"
--input '{"param": "value"}'
--output YYYY-MM-DD_results.csv --format csv

Save as JSON

node scripts/run_actor.js
--actor "apidojo~google-search-scraper"
--input '{"param": "value"}'
--output YYYY-MM-DD_results.json --format json

> `APIFY_TOKEN` must be set in environment or `.env` file.

**If Apify MCP is available:**

Tool: apify:run-actor Actor: "apidojo~google-search-scraper" Input: { "queries": ["query1", "query2"], "maxPagesPerQuery": 1, "countryCode": "us", "languageCode": "en" }


**REST API fallback:**
```bash
curl -X POST \
  "https://api.apify.com/v2/acts/apidojo~google-search-scraper/runs?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"queries": ["query1", "query2"], "maxPagesPerQuery": 1}'

Save

id

RUN_ID

. Poll until

status = SUCCEEDED

bash

curl "https://api.apify.com/v2/actor-runs/$RUN_ID?token=$APIFY_TOKEN" | grep '"status"'

Fetch results:

bash

curl "https://api.apify.com/v2/actor-runs/$RUN_ID/dataset/items?token=$APIFY_TOKEN&format=json"

node scripts/run_actor.js
--actor "apidojo~google-search-scraper"
--input '{"param": "value"}'
--output YYYY-MM-DD_results.json --format json

> `APIFY_TOKEN`必须在环境变量或`.env`文件中设置。

**如果Apify MCP可用：**

Tool: apify:run-actor Actor: "apidojo~google-search-scraper" Input: { "queries": ["query1", "query2"], "maxPagesPerQuery": 1, "countryCode": "us", "languageCode": "en" }


**REST API备选方案：**
```bash
curl -X POST \
  "https://api.apify.com/v2/acts/apidojo~google-search-scraper/runs?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"queries": ["query1", "query2"], "maxPagesPerQuery": 1}'

保存

id

为

RUN_ID

。轮询直到

status = SUCCEEDED

：

bash

curl "https://api.apify.com/v2/actor-runs/$RUN_ID?token=$APIFY_TOKEN" | grep '"status"'

获取结果：

bash

curl "https://api.apify.com/v2/actor-runs/$RUN_ID/dataset/items?token=$APIFY_TOKEN&format=json"

Step 3: Handle Edge Cases

步骤3：处理边缘情况

0 organic results: Query may trigger only ads or knowledge panels — note to user.
SERP features present (PAA, snippets): These are returned as separate result types — include them in output with
```
type
```
field.
Different results per country: Specify
```
countryCode
```
explicitly when geo-specific results are needed.

0条自然搜索结果：查询词可能仅触发广告或知识面板——需告知用户。
存在SERP特征（PAA、摘要）：这些会作为单独的结果类型返回——需在输出中包含
```
type
```
字段。
不同国家/地区结果不同：当需要特定地区的结果时，请明确指定
```
countryCode
```
。

Output Format

输出格式

undefined

undefined

Google SERP Dataset: "<query>"

Query: <query> | Country: <cc> | Pages: N | Results: N

Position	URL	Title	Snippet (truncated)	Type
1	...	...	...	organic
—	...	...	...	featured_snippet
—	...	...	...	paa

Available fields: query, position, url, title, description, type (organic/ featured_snippet/paa/local_pack/image/video), sitelinks, rating, reviewsCount

undefined

Query: <query> | Country: <cc> | Pages: N | Results: N

Position	URL	Title	Snippet (truncated)	Type
1	...	...	...	organic
—	...	...	...	featured_snippet
—	...	...	...	paa

Available fields: query, position, url, title, description, type (organic/ featured_snippet/paa/local_pack/image/video), sitelinks, rating, reviewsCount

undefined

Troubleshooting

故障排除

Google CAPTCHA / blocks: Actor handles this automatically; if persistent, reduce query batch size. Inconsistent positions: SERP positions vary by location and personalization — use

countryCode

for consistency.

Google验证码/拦截：Actor会自动处理此问题；如果问题持续，请减少查询批次大小。 排名位置不一致：SERP排名会因地理位置和个性化设置而异——使用

countryCode

确保一致性。