Web Documentation Resolver

When to Use This Skill

• Fetch and parse documentation from a URL
• Search for technical information across the web
• Build context from web sources
• Extract markdown from websites
• Query for technical documentation, APIs, or code examples

Platform Tool Mapping

webfetch

websearch

WebFetch

WebSearch

web_fetch

web_search

Cascade Resolution Strategy

For URL inputs

1. Check llms.txt first: Probe

   https://origin/llms.txt

   for site-provided structured documentation (free, always check first)
2. Fetch URL: Use platform's fetch tool to get markdown content
3. Search fallback: Use platform's search tool to find cached/mirrored versions if direct fetch fails

For query inputs

1. Search first: Use platform's search tool with relevant query (fast, free)
2. Fetch top results: Use fetch tool to get markdown from top search results if needed

Implementation

Python Script (scripts/resolve.py)

# Resolve a URL
python scripts/resolve.py "https://docs.rust-lang.org/book/"

# Resolve a query
python scripts/resolve.py "Rust async programming"

# JSON output
python scripts/resolve.py "query" --json

# Custom max chars
python scripts/resolve.py "query" --max-chars 4000

# Force specific backend
python scripts/resolve.py "query" --backend httpx

Direct Tool Usage by Platform

opencode

# Check for llms.txt
webfetch https://example.com/llms.txt

# Fetch URL
webfetch --format markdown https://docs.rust-lang.org/book/

# Search
websearch "Rust book documentation"

claude code (MCP)

# Check for llms.txt
WebFetch(url="https://example.com/llms.txt")

# Fetch URL
WebFetch(url="https://docs.rust-lang.org/book/")

# Search
WebSearch(query="Rust book documentation")

blackbox

# Check for llms.txt
web_fetch(url="https://example.com/llms.txt", prompt="Extract all content")

# Fetch URL
web_fetch(url="https://docs.rust-lang.org/book/", prompt="Extract main content")

# Search
web_search(query="Rust book documentation")

Usage Examples

Basic URL Resolution

# Using Python script (auto-detects backend)
python scripts/resolve.py "https://docs.rust-lang.org/book/"

# Or use platform tool directly
webfetch https://docs.rust-lang.org/book/  # opencode

Query Resolution

# Using Python script
python scripts/resolve.py "Rust async programming best practices 2026"

# Or use platform tool directly
websearch "Tokio runtime configuration options"  # opencode

Workflow for Building Context

1. Check for llms.txt first: Probe

   https://origin/llms.txt

2. Fetch content: Use fetch tool to get markdown from the URL
3. Search if needed: Use search tool for additional context or when fetch fails

Best Practices

• Check for llms.txt first: Many documentation sites have

  /llms.txt

  for structured content
• Use specific queries: "rust tokio spawn vs spawn_blocking difference" gets better results than "rust tokio"
• Filter by date: Add "2025" or "2026" to queries for current information
• Prefer official docs: Always check official documentation first
• Try multiple sources: If one URL fails, search for alternative mirrors

Quality Indicators

• Code examples with language markers
• API signatures and type annotations
• Configuration examples
• Version information
• Clear headings and structure

• Excessive boilerplate/navigation
• Paywall blocks
• Login requirements
• Heavy advertising

Error Handling

• Provider failures should trigger cascade fallback
• Use alternative sources when primary sources fail
• Log errors for debugging
• Fall back to search when direct fetch fails

Testing

cd .agents/skills/web-doc-resolver
python -m pytest tests/ -v

python samples/sample_basic.py
python samples/sample_json.py

Files

• scripts/resolve.py

  - Main implementation (multi-backend)

• tests/test_resolve.py

  - Unit tests

• samples/sample_basic.py

  - Basic usage examples

• samples/sample_json.py

  - JSON output examples

• reference.md

  - Detailed reference documentation