web-access Skill
First-time Installation
When users use it for the first time, follow these steps:
Step 1: Runtime Environment Detection
bash
bash ~/.claude/skills/web-access/scripts/check-deps.sh
Step 2: AI handles missing dependencies based on output
The detection script only reports facts, and installation decisions are made by AI. Install whatever is missing. If Chrome is missing, prompt the user to download it manually (auto-installation is not possible).
Step 3: After installation is complete, inform the user of the following
web-access is ready. Just tell me directly about any network-related needs, and I will automatically choose the most appropriate method:
- Only need search results → Direct search, fastest
- Need to view complete page content → Scrape page content without launching a browser
- Need login/dynamic page/browser operations → Automatically launch browser, persist login state after one login. Supports multi-sub-agent clusters using multiple browsers in parallel.
Windows users need Git Bash environment (install Git for Windows).
Browsing Philosophy
Browse like a human, not execute programs like a robot.
Humans don't list complete steps before browsing; they enter with a goal, judge as they go, solve obstacles when encountered, and dive deeper if content is insufficient — making decisions throughout centered on "what I need to get". All behaviors of this skill should follow this logic.
Three Core Judgments:
① What do I need? — Task-driven, first clarify the nature of the target information, then choose the lightest and most direct way. Don't use heavy tools for lightweight tasks, nor lightweight tools for content beyond their coverage.
② Is it enough? — If the obtained information can complete the task, it's enough. Do not over-collect, do not waste resources for the sake of "completeness". To roughly understand a video, a few frames are enough; to understand an article, reading the text is enough; there's no need to take full-page screenshots for tasks that can be done with the accessibility tree.
③ What to do when encountering obstacles? — Solve within the layer, do not backtrack, do not disturb the user. Pop-ups, login walls, ads, loading failures — judge like a human whether the obstacle really blocks the target content: handle it if it blocks, bypass it and continue if it doesn't. Only inform the user when it's confirmed that the problem cannot be solved independently. UI interaction is also an indirect layer that can be bypassed: Pagination, expansion, clicking are not the only paths to get content — content may already exist on the website, and interaction is just a means of display.
Information Acquisition Channel Selection
- Evaluate the task first, then choose the channel: Decide the starting point based on "the nature of the target information, which tool can directly obtain it", and select the lightest and most direct solution.
- Ensure information authenticity, first-hand information is better than second-hand: Search engines and aggregation platforms are information discovery entrances. When multiple search attempts don't bring qualitative improvement, upgrade to a more fundamental acquisition method: locate first-hand sources (official websites, official platforms, original pages).
| Scenario | Channel |
|---|
| Only need search summaries or keyword results, or need to discover information sources | WebSearch |
| URL is known, read page content | Jina (default, executes JS rendering at the bottom layer); use WebFetch (does not execute JS rendering) instead when needing to read structured fields in HTML source code (meta, JSON-LD, etc.) |
| URL is a PDF | Jina |
| Non-public content, or platforms where static layer is known to be ineffective (public content on Xiaohongshu, WeChat Official Accounts, etc. is also restricted by anti-scraping) | Browser CDP (direct, skips static layer) |
| Need dynamic content, login state, interactive operations, or need to freely navigate and explore in the browser like a human | Browser CDP |
Browser CDP does not require a known URL — it can start from any entry point, find target content through in-page search, clicking, jumping, etc.
Jina: Called in the format
(add prefix before URL, do not retain the original http prefix). Limited to 20 RPM, saves AI context more efficiently.
It uses Puppeteer to render pages (can handle JS/SPA) at the bottom layer, then uses the Readability algorithm to extract main article content and convert it to Markdown, filtering out noise such as navigation, ads, and sidebars. Suitable for pages centered on main text such as articles, blogs, documents, PDFs; for non-article structured pages such as video pages, data panels, product pages, it may extract incorrect blocks. After obtaining the result, judge whether the content meets the task expectations, switch the access strategy if not.
WebFetch: Directly obtains original HTML without executing JS. Structured fields such as meta and JSON-LD are usually statically embedded in HTML by the server, and WebFetch can read them directly; if the fields are dynamically injected by JS, WebFetch cannot obtain them either. Add header
Accept: text/markdown, text/html
when requesting, and websites supporting this protocol will return Markdown directly.
Selection Logic: Use Jina by default — Jina executes JS at the bottom layer and can handle dynamically rendered pages. Only use WebFetch when the task may require reading HTML source code. When neither Jina nor WebFetch can handle it (cannot obtain required information, error occurs, login required) → upgrade to browser layer.
Downgrade Forbidden: After entering a heavier channel, do not go back to use lightweight tools to complete the same target — it is equivalent to retracing a known blocked path. When encountering obstacles in the browser layer, solve them within the layer (such as handling login), instead of going back. The only exception: new sub-goals derived from browser operations can re-select channels.
After entering the browser layer, distinguish task types:
- Operational (navigation, form filling, clicking): Use accessibility tree to perceive the interface, only use screenshots for assistance when recognition fails
- Content-based (reading posts, viewing news, analyzing pages): Use accessibility tree to read text structure, and simultaneously judge whether images carry core information — extract image URLs for targeted reading if yes
Image Judgment: For social media, graphic blogs, screenshot-based content, images are considered valuable by default, actively fetch them; for tool-based, navigation-based pages, accessibility tree is considered sufficient by default.
Browser CDP Mode
Startup
bash
# Single agent task (default port 9222)
bash ~/.claude/skills/web-access/scripts/ensure-browser.sh
# Parallel tasks (port specified by main agent in task prompt, see "Parallel Research" section)
bash ~/.claude/skills/web-access/scripts/ensure-browser.sh $PORT
Output
Browser ready on port XXXX
. After startup, you must perform the following two steps:
bash
# Parse port and set session (all subsequent commands inherit automatically, no need to pass parameters repeatedly)
PORT=<Port number parsed from output>
export AGENT_BROWSER_SESSION="port-${PORT}"
Output status description:
Browser ready on port XXXX
- → Execute
bash ~/.claude/skills/web-access/scripts/close-browser.sh [port]
⚠️ Downgrade Strictly Forbidden: Only use agent-browser CDP mode, do not switch to other browser tools/MCP.
Common Commands
Record the port number from the output of
as
, use this port for subsequent commands:
bash
agent-browser --cdp $PORT open <url> # Open page
agent-browser --cdp $PORT snapshot -i # Interactive elements (for operations)
agent-browser --cdp $PORT snapshot # Complete accessibility tree (for reading text)
agent-browser --cdp $PORT click @ref-123 # Click element
agent-browser --cdp $PORT fill @ref-123 "content" # Fill input box
agent-browser --cdp $PORT wait load networkidle # Only used after navigation triggered by click/fill; open has built-in waiting, do not use after open
agent-browser --cdp $PORT scroll down 3000 # Trigger lazy loading
agent-browser --cdp $PORT screenshot /tmp/x.png
agent-browser --cdp $PORT screenshot --annotate # Upgrade solution when snapshot -i ref fails, see references/commands.md
agent-browser --cdp $PORT eval "<js>" # Execute JS, used to extract DOM information
Image Extraction
When judging that content is in images, use
to directly get image URLs from DOM, then open and take screenshots for targeted reading — much more precise than full-page screenshots.
Technical facts:
- There is a lot of content in the page that has been loaded but not displayed — images in non-current frames of carousels, text in collapsed blocks, lazy loading placeholder elements, etc., which exist in the DOM but are invisible to users. The visual layer is just a projection of DOM data. Thinking in units of data structures (containers, attributes, node relationships) can directly reach this content without relying on whether the visual layer presents it.
- Scrolling to the bottom triggers lazy loading, allowing images not in the viewport to complete loading. If you don't scroll before extracting image URLs with , some images may not have been loaded yet.
After obtaining the image URL, the Read tool natively supports reading local image files. For public image URLs that do not require a session, you can directly download them to the local and read them with Read, without going through the browser. Only images that require session/cookies need to be opened + screenshot in the browser.
Video Content Acquisition
In CDP headed mode, the browser renders realistically, and screenshots can capture the current video frame. Core capability: Seek to any time point and take screenshots, which allows discrete sampling analysis of video content.
bash
# Get total duration, formulate sampling plan
agent-browser --cdp $PORT eval "document.querySelector('video').duration"
# Seek + play + screenshot
agent-browser --cdp $PORT eval "var v=document.querySelector('video'); v.currentTime=60; v.play()"
sleep 2
agent-browser --cdp $PORT screenshot /tmp/frame.png
# Full-screen screenshots are clearer
agent-browser --cdp $PORT eval "document.querySelector('video').requestFullscreen()"
Sampling granularity (for reference only, analyze specific videos accordingly: general understanding → 30-60s intervals; understand narrative → 10s; fine-grained analysis → 1-2s) is judged based on task requirements, no need for user specification.
Login Judgment
The core question of login judgment is only one: Have I obtained the target content?
After opening the page, first try to obtain the target content and continue executing. During this process, make judgments based on two aspects of information:
- Domain Knowledge: Understanding of the website — latest timeline of X/Twitter, private content of Xiaohongshu, complete comments of Weibo, etc., such content usually requires login to obtain complete data
- Actual Page Feedback: Does the content meet expectations? Is it a downgraded version (such as popular posts instead of latest posts)? Is there obvious missing content?
Even if the page shows a login prompt, there's no need to disturb the user to log in as long as the target content has been obtained.
Only when it is confirmed that the target content cannot be obtained, infer: Can login solve this problem? If the inference is valid, inform the user:
"The target content [specific content] cannot be obtained on the current page without logging in. Please log in to [website name] in the opened Chrome window and tell me to continue after completion."
No need to restart the browser after login is completed, directly continue the original task.
Task Completion
Close the browser after the task ends (Must use this script, do not kill directly, otherwise crash windows will remain):
bash
bash ~/.claude/skills/web-access/scripts/close-browser.sh [port] # Default 9222
close-browser.sh cleans up Chrome processes and agent-browser session daemon at the same time, no additional operations are required for the caller.
Parallel Research: Sub-Agent Divide-and-Conquer Strategy
When the task contains multiple independent research objectives (such as researching N projects, N sources at the same time), it is encouraged to reasonably divide and assign to sub-agents for parallel execution, rather than the main agent processing serially.
Benefits:
- Speed: Multiple sub-agents run in parallel, total time is approximately equal to the duration of a single sub-task
- Context Protection: Scraped content does not enter the main agent's context, the main agent only receives summaries, saving tokens
Sub-Agents Need to Inherit the Skill:
Write
Follow the guidelines of web-access skill
in the sub-agent prompt, and the sub-agent will automatically load the skill, no need to copy skill content or specify paths in the prompt.
Sub-Agent Prompt Writing: Goal-oriented, not step-by-step instructions
Sub-agents have complete skill knowledge and independent judgment capabilities. The main agent's responsibility is to clearly state what is needed, and only limit how to do it when necessary and confident. Over-specifying steps will deprive sub-agents of judgment space, and instead introduce assumption errors from the main agent.
Wrong: Over-specification (pre-filling unverified URL/account):
Open https://x.com/SomeAccount, scrape the latest tweets
Correct: Goal-oriented (sub-agent discovers path independently):
Find the official X account of XX, get the latest tweet content
Key principle: Do not pre-fill unverified information. Information can be directly passed in only when it is provided directly by the user or has been confirmed.
Divide-and-Conquer Judgment Criteria:
| Suitable for Divide-and-Conquer | Not Suitable for Divide-and-Conquer |
|---|
| Goals are independent, results do not depend on each other | Goals have dependencies, the next one needs the result of the previous one |
| Each sub-task has sufficient volume (multi-page scraping, multi-round searching) | Simple single-page query, divide-and-conquer overhead is greater than benefit |
| Requires CDP browser or long-running tasks | Lightweight queries that can be completed with a few WebSearch / Jina |
CDP Concurrency: Each port starts an independent Chrome instance, no mutual interference.
When the main agent starts sub-agents, explicitly specify an exclusive port for each sub-agent in the task prompt to avoid browser instance conflicts (select from 9222 to 9299, each sub-agent uses a different port). After receiving the port, the sub-agent calls
to start.
Special Task Rules
Verification Tasks
The goal of verification is first-hand sources, not more second-hand reports — multiple media citing the same error will create an illusion of circular confirmation.
Search is used for locating sources, not for proving authenticity. After finding the source, directly access and read the original text.
| Information Type | First-hand Source |
|---|
| Policies/Regulations | Official website of the issuing authority |
| Corporate Announcements | Official news page of the company |
| Academic Statements | Original papers/institutional official websites |
When official website cannot be found: Original reports from authoritative media (non-reprinted) can be used as secondary basis, but need to inform the user: "Official original text not found, the following verification comes from [media name] report, there may be paraphrasing errors."
Understanding Tool Capability Boundaries
When in doubt about the capabilities of any tool (MCP, CLI, library), if there is not enough knowledge to be sure, check the official documentation first; if there is not enough documentation, consider viewing the source code before making a judgment, do not guess, do not transfer uncertainty to users.
References Index
| File | When to Load |
|---|
| When infrequently used commands are needed (drag, storage, pdf, etc.) |
| When details of login process need to be understood |