Search Layer v2.2 — Intent-Aware Multi-Source Retrieval Protocol

Four parallel sources: Brave (

web_search

) + Exa + Tavily + Grok. Automatically select strategies, adjust weights, and synthesize results based on intent.

Execution Flow

用户查询
    ↓
[Phase 1] 意图分类 → 确定搜索策略
    ↓
[Phase 2] 查询分解 & 扩展 → 生成子查询
    ↓
[Phase 3] 多源并行检索 → Brave + search.py (Exa + Tavily + Grok)
    ↓
[Phase 4] 结果合并 & 排序 → 去重 + 意图加权评分
    ↓
[Phase 5] 知识合成 → 结构化输出

Phase 1: Intent Classification

After receiving a search request, first determine the intent type, then decide the search strategy. Do not ask users which mode to use.

Intent	Recognition Signals	Mode	Freshness	Weight Bias
Factual	"what is X", "definition of X", "What is X"	answer	—	Authority 0.5
Status	"latest progress of X", "current status of X", "latest X"	deep	pw/pm	Freshness 0.5
Comparison	"X vs Y", "difference between X and Y"	deep	py	Keyword 0.4 + Authority 0.4
Tutorial	"how to do X", "X tutorial", "how to X"	answer	py	Authority 0.5
Exploratory	"learn X in depth", "X ecosystem", "about X"	deep	—	Authority 0.5
News	"X news", "X this week", "X this week"	deep	pd/pw	Freshness 0.6
Resource	"X official website", "X GitHub", "X documentation"	fast	—	Keyword 0.5

See
references/intent-guide.md
for detailed classification guidelines

Judgment Rules:

Scan for signal words in the query
Select the most specific type when multiple types match
Default to
```
exploratory
```
if unable to judge

Phase 2: Query Decomposition & Expansion

Expand the user query into a set of sub-queries based on the intent type:

General Rules

Automatic technical synonym expansion: k8s→Kubernetes, JS→JavaScript, Go→Golang, Postgres→PostgreSQL
Chinese technical queries: generate English variants simultaneously (e.g. "Rust 异步编程" → additionally search for "Rust async programming")

Expansion by Intent

Intent	Expansion Strategy	Example
Factual	Add "definition", "explained"	"WebTransport" → "WebTransport", "WebTransport explained overview"
Status	Add year, "latest", "update"	"Deno progress" → "Deno 2.0 latest 2026", "Deno update release"
Comparison	Split into 3 sub-queries	"Bun vs Deno" → "Bun vs Deno", "Bun advantages", "Deno advantages"
Tutorial	Add "tutorial", "guide", "step by step"	"Rust CLI" → "Rust CLI tutorial", "Rust CLI guide step by step"
Exploratory	Split into 2-3 perspectives	"RISC-V" → "RISC-V overview", "RISC-V ecosystem", "RISC-V use cases"
News	Add "news", "announcement", date	"AI news" → "AI news this week 2026", "AI announcement latest"
Resource	Add specific resource type	"Anthropic MCP" → "Anthropic MCP official documentation"

Phase 3: Multi-source Parallel Retrieval

Step 1: Brave (All Modes)

Call

web_search

for each sub-query. If the intent has freshness requirements, pass the

freshness

parameter:

web_search(query="Deno 2.0 latest 2026", freshness="pw")

Step 2: Exa + Tavily + Grok (Deep / Answer Mode)

Call search.py for sub-queries, pass intent and freshness:

bash

python3 /home/node/.openclaw/workspace/skills/search-layer/scripts/search.py \
  --queries "子查询1" "子查询2" "子查询3" \
  --mode deep \
  --intent status \
  --freshness pw \
  --num 5

Source Participation Matrix by Mode:

Mode	Exa	Tavily	Grok	Description
fast	✅	❌	fallback	Exa first; use Grok when no Exa key is available
deep	✅	✅	✅	Three sources run in parallel
answer	❌	✅	❌	Tavily only (includes AI answer)

Parameter Description:

Parameter	Description
`--queries`	Multiple sub-queries executed in parallel (single query can also be passed as positional parameter)
`--mode`	fast / deep / answer
`--intent`	Intent type, affects scoring weight (no scoring if not passed, behavior consistent with v1)
`--freshness`	pd(24h) / pw(week) / pm(month) / py(year)
`--domain-boost`	Comma separated domain names, matching results get +0.2 authority score
`--num`	Number of results per source per query

Grok Source Description:

Call Grok model (
```
grok-4.1-fast
```
) via completions API, return structured search results using its real-time knowledge
Automatically detect time-sensitive queries and inject current time context
Run in parallel with Exa and Tavily in deep mode

Need to configure Grok's

apiUrl

apiKey

model

~/.openclaw/credentials/search.json

(or via environment variables

GROK_API_URL

GROK_API_KEY

GROK_MODEL

)

Automatically downgrade to Exa + Tavily dual sources if Grok configuration is missing

Step 3: Merge

Merge Brave results with search.py output. Deduplicate by canonical URL, mark sources.

If search.py returns the

score

field, use it for sorting; if Brave results have no score, calculate it using the same intent weight formula.

Phase 3.5: Reference Tracking (Thread Pulling)

When search results include GitHub issue/PR links and the intent is Status or Exploratory, automatically trigger reference tracking.

Automatic Trigger Conditions

Intent is
```
status
```
or
```
exploratory
```

Search results include

github.com/.../issues/

github.com/.../pull/

URLs

Method 1: search.py --extract-refs (Batch)

Extract reference graph directly from search results without additional calls:

bash

python3 search.py "OpenClaw config validation bug" --mode deep --intent status --extract-refs

There will be an additional

refs

field in the output, containing the reference list for each result URL.

You can also skip searching and extract references directly for known URLs:

bash

python3 search.py --extract-refs-urls "https://github.com/owner/repo/issues/123" "https://github.com/owner/repo/issues/456"

Method 2: fetch-thread (Single URL Deep Crawl)

Pull complete discussion stream + structured references for a single URL:

bash

python3 fetch_thread.py "https://github.com/owner/repo/issues/123" --format json
python3 fetch_thread.py "https://github.com/owner/repo/issues/123" --format markdown
python3 fetch_thread.py "https://github.com/owner/repo/issues/123" --extract-refs-only

GitHub scenario (issue/PR): Pull body + all comments + timeline events (cross-references, commits) via API, extract:

Issue/PR references (#123, owner/repo#123)
Duplicate markers
Commit references
Associated PR/issue (timeline cross-references)
External URLs

General web scenario: web fetch + regex extraction of reference links.

Agent Execution Flow

Step 1: search-layer search → Get initial results
Step 2: search.py --extract-refs or fetch-thread → Extract clue graph
Step 3: Agent filters high-value clues (LLM judges which are worth tracking)
Step 4: fetch-thread deeply crawls each high-value clue
Step 5: Repeat Step 2-4 until information is closed or depth limit is reached (recommended max_depth=3)

Phase 4: Result Sorting

Scoring Formula

score = w_keyword × keyword_match + w_freshness × freshness_score + w_authority × authority_score

Weights are determined by intent (see Phase 1 table). Each item:

keyword_match (0-1): Coverage of query terms in title + abstract
freshness_score (0-1): Based on publication date, newer is higher (no date = 0.5)
authority_score (0-1): Based on domain authority level
- Tier 1 (1.0): github.com, stackoverflow.com, official documentation sites
- Tier 2 (0.8): HN, dev.to, well-known technical blogs
- Tier 3 (0.6): Medium, Juejin, InfoQ
- Tier 4 (0.4): Others

See
references/authority-domains.json
for full domain rating table

Domain Boost

Manually specify domains to be weighted via the

--domain-boost

parameter (matching results get +0.2 authority score):

bash

search.py "query" --mode deep --intent tutorial --domain-boost dev.to,freecodecamp.org

Recommended combinations:

Tutorial →

dev.to, freecodecamp.org, realpython.com, baeldung.com

Resource →
```
github.com
```

News →

techcrunch.com, arstechnica.com, theverge.com

Phase 5: Knowledge Synthesis

Select synthesis strategy based on the number of results:

Small result set (≤5 items)

Display one by one, each with source tag and score:

1. [Title](url) — snippet... `[brave, exa]` ⭐0.85
2. [Title](url) — snippet... `[tavily]` ⭐0.72

Medium result set (5-15 items)

Cluster by theme + summary per group:

**Theme A: [Description]**
- [Result 1] — Key points... `[source]`
- [Result 2] — Key points... `[source]`

**Theme B: [Description]**
- [Result 3] — Key points... `[source]`

Large result set (15+ items)

High-level overview + Top 5 + in-depth prompt:

[An overview summarizing the main findings]

**Top 5 most relevant results:**
1. ...
2. ...

Found N results in total, covering [source list]. Which aspect do you want to explore in depth?

Synthesis Rules

Give the answer first, then list sources (don't start with "I searched for...")
Aggregate by theme, not by source (don't do "Brave results: ... Exa results: ...")
Explicitly mark conflicting information: clearly point out when different sources have contradictory statements
Confidence expression:
- Multi-source consistent + fresh → state directly
- Single source or older → "According to [source], ..."
- Conflicting or uncertain → "There are different opinions: A believes..., B believes..."

Degradation Strategy

Exa 429/5xx → continue with Brave + Tavily + Grok
Tavily 429/5xx → continue with Brave + Exa + Grok
Grok timeout/error → continue with Brave + Exa + Tavily
Overall search.py failure → use only Brave
```
web_search
```
(always available)
Never block the main process because of a single source failure

Backward Compatibility

When the

--intent

parameter is not included, the behavior of search.py is exactly the same as v1 (no scoring, output in original order).

Existing callers (e.g. github-explorer) do not need modification.

Quick Reference

Scenario	Command
Quick fact check	`web_search` + `search.py --mode answer --intent factual`
In-depth research	`web_search` + `search.py --mode deep --intent exploratory`
Latest updates	`web_search(freshness="pw")` + `search.py --mode deep --intent status --freshness pw`
Comparative analysis	`web_search` × 3 queries + `search.py --queries "A vs B" "A pros" "B pros" --intent comparison`
Find resources	`web_search` + `search.py --mode fast --intent resource`

search-layer

NPX Install

Tags

SKILL.md Content (Chinese)

Search Layer v2.2 — Intent-Aware Multi-Source Retrieval Protocol

Execution Flow

Phase 1: Intent Classification

Phase 2: Query Decomposition & Expansion

General Rules

Expansion by Intent

Phase 3: Multi-source Parallel Retrieval

Step 1: Brave (All Modes)

Step 2: Exa + Tavily + Grok (Deep / Answer Mode)

Step 3: Merge

Phase 3.5: Reference Tracking (Thread Pulling)

Automatic Trigger Conditions

Method 1: search.py --extract-refs (Batch)

Method 2: fetch-thread (Single URL Deep Crawl)

Agent Execution Flow

Phase 4: Result Sorting

Scoring Formula

Domain Boost

Phase 5: Knowledge Synthesis

Small result set (≤5 items)

Medium result set (5-15 items)

Large result set (15+ items)

Synthesis Rules

Degradation Strategy

Backward Compatibility

Quick Reference