Back to Details

web-access

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Web access

Web 访问

One treg key reads the web: free search and page fetch first, and the rest of the catalog behind them for the pages and platforms a free fetch can't reach. Nothing runs on this machine: no browser, no cookies, no burner accounts. treg calls the vendor and, on a routed job, tries the next one when a vendor comes back empty or blocked.
Runs on treg. If it isn't set up (
treg --version
fails), read https://treg.to/llms.txt and follow it to install and sign in, then come back here.
一个 treg key 即可读取 Web：优先使用免费搜索和页面读取；对于免费读取无法覆盖的页面和平台，则使用目录中的其他服务。没有任何操作会在本机运行：无需浏览器、Cookie 或临时账号。treg 会调用供应商；在路由任务中，当某个供应商返回空结果或请求被拦截时，会尝试下一家。
在 treg 上运行。如果尚未配置好（运行 
treg --version
失败），请阅读 https://treg.to/llms.txt，并按照其中的说明完成安装和登录，然后返回此处。

Pick the job

选择任务

Know the URL?             → read it (free first)
Need to find pages?       → search (free) → read the 3–5 best results
A platform object?        → never scrape its HTML: use the platform's job
  (post, profile, video,     structured fields, fewer tokens, survives redesigns
   product, review, place)
A whole site?             → map it → read only the pages that matter
Need typed fields?        → structured extract with a JSON schema or an instruction
Need it recent or dated?  → news search, not web search
Know the URL?             → read it (free first)
Need to find pages?       → search (free) → read the 3–5 best results
A platform object?        → never scrape its HTML: use the platform's job
  (post, profile, video,     structured fields, fewer tokens, survives redesigns
   product, review, place)
A whole site?             → map it → read only the pages that matter
Need typed fields?        → structured extract with a JSON schema or an instruction
Need it recent or dated?  → news search, not web search

Free first

免费优先

Web search and page reads are free through TinyFish:
  • treg call treg.web.search --data '{"q":"…"}' --header 'X-Treg-Route-Prefer: tinyfish'
  • treg call treg.web.extract --data '{"url":"…"}' --header 'X-Treg-Route-Prefer: tinyfish'
Both are routed: when TinyFish comes back empty or blocked, treg tries the next vendor (Crawl4AI, AnyAPI, …). You pay the vendor that answered plus any vendor before it that bills a miss (
treg catalog get
says how many do); 
X-Treg-Route-Max-Cost
caps the walk. To read up to ten URLs in one free call, call 
tinyfish.web.fetch
directly with 
{"urls":[…]}
; it does not fall back.
Platform data (social, stores, reviews, maps) is not free: cents or fractions of a cent a call. Say so before a batch.
通过 TinyFish 可以免费进行 Web 搜索和页面读取：
  • treg call treg.web.search --data '{"q":"…"}' --header 'X-Treg-Route-Prefer: tinyfish'
  • treg call treg.web.extract --data '{"url":"…"}' --header 'X-Treg-Route-Prefer: tinyfish'
两者都支持路由：当 TinyFish 返回空结果或请求被拦截时，treg 会尝试下一家供应商（Crawl4AI、AnyAPI 等）。你只需为提供答案的供应商付费，也可能需要为此前会为未命中调用计费的供应商付费（
treg catalog get
会显示有多少家会如此计费）；
X-Treg-Route-Max-Cost
可限制整条供应商回退链的成本。若要在一次免费调用中读取最多十个 URL，可直接调用 
tinyfish.web.fetch
并传入 
{"urls":[…]}
；该调用不会回退。
平台数据（社交平台、商店、评论和地图）并非免费：每次调用需支付数美分或几分之一美分。执行批量任务前应先告知用户这一点。

Jobs

任务

Search the catalog for the words in the second column. The vendors are examples of who answers today; take whatever the catalog ranks best now.
You wantSearch the catalog forExample vendorsNotes
Web results
treg.web.search
TinyFish, Exa, Perplexity, Keenablefree with the TinyFish header
A page as markdown
treg.web.extract
TinyFish, Crawl4AI, AnyAPIfree first; blocked pages fall back
Typed fields from a page
treg.web.extract.structured
send 
url
plus a 
schema
and/or 
instruction
Every URL on a site
treg.web.map
AnyAPI, Firecrawl, Tavilyadd 
q
to rank URLs for a topic
Many pages of a site"crawl a website"Firecrawl, Crawl4AI, Olostepmap first when you need only some pages
Recent news
treg.web.search.news
Serper, DataForSEO, Exadated results
Papers"search research publications"TinyFish, Exa, Serper
X, LinkedIn, Reddit, TikTok, Instagram, Threads, Facebook"x search posts", "linkedin company posts", "reddit subreddit posts", "tiktok video comments", …TikHub, ScrapeCreators, HarvestAPI, AnyAPIprofiles, posts, comments, reactions as fields
YouTube, Bilibili"youtube video comments", "youtube search videos"Crawl4AI, TikHub, ScrapeCreators
Xiaohongshu, Douyin, Weibo"xiaohongshu user notes", "douyin search videos"TikHub, JustOneAPI
Stores"amazon product detail", "amazon search products", "walmart search", "ebay search"JustOneAPI, SerpAPI, Crawl4AIprice, rating, reviews as fields
Reviews"app store reviews", "google play app", "trustpilot reviews", "glassdoor reviews", "google maps reviews"Crawl4AI, Bright Data, LiteScrape
Places"google maps search", "local results"DataForSEO, SerpAPI
Google results as Google shows them"google serp organic"DataForSEO, Serperranks, snippets, features
在目录中搜索第二列中的关键词。表中的供应商只是当前可能提供结果的示例；应始终选择目录当前排名最靠前的可用项。
你想要在目录中搜索示例供应商备注
Web 搜索结果
treg.web.search
TinyFish, Exa, Perplexity, Keenable使用 TinyFish 请求头时免费
Markdown 格式的页面
treg.web.extract
TinyFish, Crawl4AI, AnyAPI免费优先；受阻页面自动回退
页面中的特定类型字段
treg.web.extract.structured
传入 
url
以及 
schema
和/或 
instruction
站点上的所有 URL
treg.web.map
AnyAPI, Firecrawl, Tavily添加 
q
，按主题对 URL 进行排序
站点中的多个页面"crawl a website"Firecrawl, Crawl4AI, Olostep只需要部分页面时先执行映射
最新新闻
treg.web.search.news
Serper, DataForSEO, Exa带日期的结果
论文"search research publications"TinyFish, Exa, Serper
X、LinkedIn、Reddit、TikTok、Instagram、Threads、Facebook"x search posts", "linkedin company posts", "reddit subreddit posts", "tiktok video comments", …TikHub, ScrapeCreators, HarvestAPI, AnyAPI个人资料、帖子、评论和互动均作为字段返回
YouTube、Bilibili"youtube video comments", "youtube search videos"Crawl4AI, TikHub, ScrapeCreators
小红书、抖音、微博"xiaohongshu user notes", "douyin search videos"TikHub, JustOneAPI
商店"amazon product detail", "amazon search products", "walmart search", "ebay search"JustOneAPI, SerpAPI, Crawl4AI价格、评分和评论均作为字段返回
评论"app store reviews", "google play app", "trustpilot reviews", "glassdoor reviews", "google maps reviews"Crawl4AI, Bright Data, LiteScrape
地点"google maps search", "local results"DataForSEO, SerpAPI
Google 原样展示的结果"google serp organic"DataForSEO, Serper排名、摘要和功能模块

Finding the endpoint

查找 endpoint

The catalog changes every week, so never trust a remembered endpoint id.
  1. treg catalog search "<the job in plain words>"
    , not a vendor name.
  2. Prefer a routed 
    treg.*
    endpoint when there is one: the team's own keys first, then vendors in order, and most misses are free.
  3. treg catalog get <id>
    : the price, the vendors behind a routed job, and each one's measured hit rate (it found something), not just works (it returned 200), and speed.
  4. One small call first. Read 
    _treg.served_by
    , 
    _treg.tried
    and the reported charge, then scale.
  5. Nothing fits: 
    treg catalog request "<what is missing>"
    .
目录每周都会变化，因此绝不要依赖记忆中的 endpoint id。
  1. treg catalog search "<the job in plain words>"
    ，不要使用供应商名称。
  2. 如果存在支持路由的 
    treg.*
    endpoint，应优先使用：先使用团队自己的 key，再按顺序使用各供应商，而且大多数未命中调用都是免费的。
  3. treg catalog get <id>
    ：查看价格、路由任务背后的供应商，以及每家供应商经测量的命中率（确实找到了内容），而不仅仅是可用率（返回了 200），同时还可以查看速度。
  4. 先发起一次小规模调用。读取 
    _treg.served_by
    、
    _treg.tried
    和报告的费用，然后扩大调用规模。
  5. 没有合适选项时：
    treg catalog request "<what is missing>"
    。

Steering a routed job

控制路由任务

Options ride as headers, never in the body:
HeaderUse it to
X-Treg-Route-Prefer: <vendor>
try that vendor first (free TinyFish, or the one that worked last time)
X-Treg-Route-Exclude: <vendor>
skip a vendor that gave you bad answers
X-Treg-Route-Max-Cost: <usd>
cap one call; uncapped, a miss can walk up to $1 of vendors
X-Treg-Route-Waterfall: 0
stop at the first miss when only one vendor's answer will do
Batches: always send a 
Max-Cost
.
这些选项通过请求头传递，绝不能放在请求体中：
Header用途
X-Treg-Route-Prefer: <vendor>
优先尝试该供应商（TinyFish 免费服务，或上次奏效的供应商）
X-Treg-Route-Exclude: <vendor>
跳过提供糟糕结果的供应商
X-Treg-Route-Max-Cost: <usd>
限制单次调用的成本；不设上限时，一次未命中可能依次尝试成本高达 1 美元的供应商
X-Treg-Route-Waterfall: 0
只需要一家供应商的结果时，在第一次未命中后停止
批量任务：始终发送 
Max-Cost
。

Habits that save money and tokens

节省费用和 tokens 的习惯

  • Read pages as markdown, and cut navigation and footers before they reach the context.
  • A platform job returns fields; a scraped page of the same thing costs several times the tokens.
  • Search, then read only the results you need. Never read a whole site to answer one question.
  • Keep the source URL next to every fact you hand back.
  • 以 markdown 形式读取页面，并在导航栏和页脚进入上下文前将其移除。
  • 平台任务返回字段；抓取同一内容的页面会消耗数倍的 tokens。
  • 先搜索，再只读取需要的结果。绝不要为了回答一个问题而读取整个站点。
  • 返回每项事实时，都将来源 URL 放在其旁边。

Recipes

示例

  • Research a company: web search → read its site, pricing and docs pages → news search for the last 90 days → its LinkedIn company profile.
  • Read a docs site into context: map the site with 
    q
    set to the topic → read the top pages.
  • Watch a page for changes: read it, keep a hash of the text, read it again next run and report the diff. Schedule it with the user's agent (Claude Code 
    /schedule
    or 
    /loop
    , cron for others).
  • Social listening: search posts on X, Reddit and LinkedIn → read the comments on the best ones → summarise by theme with links.
  • Compare a product across stores: search it on each store → product detail for the matches → one table of price, rating and review count.
  • Collect reviews: app store and Google Play for apps, Trustpilot and Glassdoor for companies, Google Maps for places → quote the patterns, with counts.
  • Videos into notes: YouTube search → video detail and comments → summarise.
  • 研究一家公司： Web 搜索 → 阅读其网站、定价和文档页面 → 搜索过去 90 天的新闻 → 查看其 LinkedIn 公司资料。
  • 将文档站点读入上下文： 使用 
    q
    设置主题来映射站点 → 读取排名最前的页面。
  • 监控页面变化： 读取页面，保留文本的哈希值，下次运行时再次读取并报告差异。使用用户的 agent 进行调度（Claude Code 
    /schedule
    或 
    /loop
    ，其他情况使用 cron）。
  • 社交舆情监测： 搜索 X、Reddit 和 LinkedIn 上的帖子 → 阅读最佳帖子下的评论 → 按主题汇总并附上链接。
  • 跨商店比较产品： 在每个商店中搜索该产品 → 获取匹配项的产品详情 → 汇总为一张包含价格、评分和评论数量的表格。
  • 收集评论： 针对应用查询 App Store 和 Google Play，针对公司查询 Trustpilot 和 Glassdoor，针对地点查询 Google Maps → 引用这些评论中的共性，并附上数量。
  • 将视频转为笔记： YouTube 搜索 → 视频详情和评论 → 汇总。

Check the setup

检查设置

treg --version && treg balance
treg call treg.web.search --data '{"q":"treg web access"}' --header 'X-Treg-Route-Prefer: tinyfish'
A result with 
_treg.served_by
and no charge means free search works. Something wrong: 
treg feedback "<what happened>"
reaches the team.
treg --version && treg balance
treg call treg.web.search --data '{"q":"treg web access"}' --header 'X-Treg-Route-Prefer: tinyfish'
若结果中包含 
_treg.served_by
且没有费用，说明免费搜索可用。出现问题时：
treg feedback "<what happened>"
可将问题反馈给团队。

Lines not to cross

不可逾越的红线

Public data only. Never log in as the user, ask for their cookies or get round a paywall. Don't bulk collect personal data about private individuals. When a vendor refuses a page, take the fallback or tell the user; don't hammer it.
仅限公开数据。绝不以用户身份登录、索要用户的 Cookie 或绕过付费墙。不要批量收集非公开人士的个人数据。当供应商拒绝访问某个页面时，采用回退方案或告知用户；不要反复请求该供应商。