web-access

Original：🇺🇸 English

Translated

Use whenever a task needs something from the live web — search the web or the news, read a URL, map or crawl a site, pull structured fields from a page, or get data from a platform (X, LinkedIn, Reddit, TikTok, Instagram, YouTube, Xiaohongshu, Amazon, app stores, Google Maps, papers). Picks the right tool for the job, starts free, and lets treg fall back to the next vendor when one fails. No local scrapers, browsers or cookies.

2installs

Sourcesuperdesigndev/treg

Added on2026-10-17

NPX Install

npx skill4agent add superdesigndev/treg web-access

SKILL.md Content

View Translation Comparison →

Web access

One treg key reads the web: free search and page fetch first, and the rest of the catalog behind them for the pages and platforms a free fetch can't reach. Nothing runs on this machine: no browser, no cookies, no burner accounts. treg calls the vendor and, on a routed job, tries the next one when a vendor comes back empty or blocked.

Runs on treg. If it isn't set up (

treg --version

fails), read https://treg.to/llms.txt and follow it to install and sign in, then come back here.

Pick the job

Know the URL?             → read it (free first)
Need to find pages?       → search (free) → read the 3–5 best results
A platform object?        → never scrape its HTML: use the platform's job
  (post, profile, video,     structured fields, fewer tokens, survives redesigns
   product, review, place)
A whole site?             → map it → read only the pages that matter
Need typed fields?        → structured extract with a JSON schema or an instruction
Need it recent or dated?  → news search, not web search

Free first

Web search and page reads are free through TinyFish:

treg call treg.web.search --data '{"q":"…"}' --header 'X-Treg-Route-Prefer: tinyfish'

treg call treg.web.extract --data '{"url":"…"}' --header 'X-Treg-Route-Prefer: tinyfish'

Both are routed: when TinyFish comes back empty or blocked, treg tries the next vendor (Crawl4AI, AnyAPI, …). You pay the vendor that answered plus any vendor before it that bills a miss (

treg catalog get

says how many do);

X-Treg-Route-Max-Cost

caps the walk. To read up to ten URLs in one free call, call

tinyfish.web.fetch

directly with

{"urls":[…]}

; it does not fall back.

Platform data (social, stores, reviews, maps) is not free: cents or fractions of a cent a call. Say so before a batch.

Jobs

Search the catalog for the words in the second column. The vendors are examples of who answers today; take whatever the catalog ranks best now.

You want	Search the catalog for	Example vendors	Notes
Web results	`treg.web.search`	TinyFish, Exa, Perplexity, Keenable	free with the TinyFish header
A page as markdown	`treg.web.extract`	TinyFish, Crawl4AI, AnyAPI	free first; blocked pages fall back
Typed fields from a page	`treg.web.extract.structured`		send `url` plus a `schema` and/or `instruction`
Every URL on a site	`treg.web.map`	AnyAPI, Firecrawl, Tavily	add `q` to rank URLs for a topic
Many pages of a site	"crawl a website"	Firecrawl, Crawl4AI, Olostep	map first when you need only some pages
Recent news	`treg.web.search.news`	Serper, DataForSEO, Exa	dated results
Papers	"search research publications"	TinyFish, Exa, Serper
X, LinkedIn, Reddit, TikTok, Instagram, Threads, Facebook	"x search posts", "linkedin company posts", "reddit subreddit posts", "tiktok video comments", …	TikHub, ScrapeCreators, HarvestAPI, AnyAPI	profiles, posts, comments, reactions as fields
YouTube, Bilibili	"youtube video comments", "youtube search videos"	Crawl4AI, TikHub, ScrapeCreators
Xiaohongshu, Douyin, Weibo	"xiaohongshu user notes", "douyin search videos"	TikHub, JustOneAPI
Stores	"amazon product detail", "amazon search products", "walmart search", "ebay search"	JustOneAPI, SerpAPI, Crawl4AI	price, rating, reviews as fields
Reviews	"app store reviews", "google play app", "trustpilot reviews", "glassdoor reviews", "google maps reviews"	Crawl4AI, Bright Data, LiteScrape
Places	"google maps search", "local results"	DataForSEO, SerpAPI
Google results as Google shows them	"google serp organic"	DataForSEO, Serper	ranks, snippets, features

Finding the endpoint

The catalog changes every week, so never trust a remembered endpoint id.

treg catalog search "<the job in plain words>"

, not a vendor name.

Prefer a routed
```
treg.*
```
endpoint when there is one: the team's own keys first, then vendors in order, and most misses are free.
```
treg catalog get <id>
```
: the price, the vendors behind a routed job, and each one's measured hit rate (it found something), not just works (it returned 200), and speed.
One small call first. Read
```
_treg.served_by
```
,
```
_treg.tried
```
and the reported charge, then scale.

Nothing fits:

treg catalog request "<what is missing>"

.

Steering a routed job

Options ride as headers, never in the body:

Header	Use it to
`X-Treg-Route-Prefer: <vendor>`	try that vendor first (free TinyFish, or the one that worked last time)
`X-Treg-Route-Exclude: <vendor>`	skip a vendor that gave you bad answers
`X-Treg-Route-Max-Cost: <usd>`	cap one call; uncapped, a miss can walk up to $1 of vendors
`X-Treg-Route-Waterfall: 0`	stop at the first miss when only one vendor's answer will do

Batches: always send a

Max-Cost

.

Habits that save money and tokens

Read pages as markdown, and cut navigation and footers before they reach the context.
A platform job returns fields; a scraped page of the same thing costs several times the tokens.
Search, then read only the results you need. Never read a whole site to answer one question.
Keep the source URL next to every fact you hand back.

Recipes

Research a company: web search → read its site, pricing and docs pages → news search for the last 90 days → its LinkedIn company profile.
Read a docs site into context: map the site with
```
q
```
set to the topic → read the top pages.
Watch a page for changes: read it, keep a hash of the text, read it again next run and report the diff. Schedule it with the user's agent (Claude Code
```
/schedule
```
or
```
/loop
```
, cron for others).
Social listening: search posts on X, Reddit and LinkedIn → read the comments on the best ones → summarise by theme with links.
Compare a product across stores: search it on each store → product detail for the matches → one table of price, rating and review count.
Collect reviews: app store and Google Play for apps, Trustpilot and Glassdoor for companies, Google Maps for places → quote the patterns, with counts.
Videos into notes: YouTube search → video detail and comments → summarise.

Check the setup

treg --version && treg balance
treg call treg.web.search --data '{"q":"treg web access"}' --header 'X-Treg-Route-Prefer: tinyfish'

A result with

_treg.served_by

and no charge means free search works. Something wrong:

treg feedback "<what happened>"

reaches the team.

Lines not to cross

Public data only. Never log in as the user, ask for their cookies or get round a paywall. Don't bulk collect personal data about private individuals. When a vendor refuses a page, take the fallback or tell the user; don't hammer it.