agentic-browser-testing

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

<objective> A scripted Playwright test breaks the moment a button moves or a class renames; writing one for a dashboard that changes weekly is a maintenance treadmill. This skill stands up a goal-driven browser agent instead: it reads a natural-language goal, explores the app via the accessibility tree (Playwright MCP `browser_snapshot`), and asserts the outcome against an explicit oracle. The failure mode it prevents is the one that makes teams distrust agents — an agent that reports "success" while stuck on the login page because nothing forced it to prove where it landed. You leave with a deterministic, CI-gated agent run and a graduation path to a durable scripted test once the flow stabilizes. </objective>

<objective> 脚本化Playwright测试会在按钮位置变动或类名修改时立即失效；为每周更新的仪表盘编写这类测试会陷入无休止的维护循环。本技能提供了一种替代方案：搭建基于目标的浏览器Agent，它读取自然语言目标，通过无障碍树（Playwright MCP的`browser_snapshot`接口）探索应用，并依据明确的验证规则（oracle）验证结果。它能避免一种让团队不信任Agent的失效场景：Agent卡在登录页面却报告「成功」，因为没有强制要求它证明最终到达的位置。你将获得具备确定性、受CI门禁管控的Agent测试运行流程，以及当测试流程稳定后，将其转换为持久化脚本化测试的升级路径。 </objective>

Quick Route

快速指引

Situation	Go to
Stand up a goal-driven run from scratch	Discovery + `references/setup.md`
Decide agentic vs scripted for a given flow	Fit: Intent-Driven vs Scripted
Agent passes one run, fails the next	Determinism
"How does it click without screenshots?"	Interaction Model
Runs are slow / burning tokens	Cost and Latency
Agent reports false success	Success Assertion (the Oracle)
Flow is stable — make it permanent	Graduation → `references/graduation-and-ci.md`
Block a merge on the goal	CI Gating → `references/graduation-and-ci.md`
Canvas / no accessibility tree	Canvas Fallback → `references/graduation-and-ci.md`

场景	查看内容
从零搭建基于目标的测试运行流程	探索问题 + `references/setup.md`
为特定流程选择Agent化还是脚本化测试	适配性：意图驱动 vs 脚本化
Agent测试一次通过、下次失败	确定性配置
「无需截图它如何实现点击？」	交互模型
测试运行缓慢 / 消耗大量token	成本与延迟控制
Agent报告虚假成功	成功断言（验证规则）
流程已稳定——转为持久化测试	升级路径 → `references/graduation-and-ci.md`
基于测试目标阻止代码合并	CI门禁 → `references/graduation-and-ci.md`
画布应用 / 无无障碍树	画布 fallback 方案 → `references/graduation-and-ci.md`

Discovery Questions

探索问题

First, check

.agents/qa-project-context.md

in the project root and skip anything it already answers (stack, environments, seed/reset tooling, model access).

Which flow, and how often does its UI change? Fast-changing/experimental UI favors intent-driven; a stable critical path (login) favors scripted. This decides the whole approach.
Is there a seeded fixture and a way to reset state? Determinism is impossible without seeded data and a per-run reset. If neither exists, that is step zero.
Can you deep-link past auth to a seeded entry point? Re-driving login every run is the biggest avoidable cost; a seeded entry URL scopes the goal and cuts steps.
What is the unambiguous success oracle? Specific account text, a
```
/dashboard
```
URL, an order number — plus a forbidden state. "No error" is not an oracle.
Does the target render to canvas / WebGL? No accessibility tree means snapshot-first won't work; plan the vision fallback or instrument the canvas with ARIA.
Which model and budget? Pin a model id and a step budget up front; tier cheap steps to Haiku 4.5 / Sonnet 4.6 and reserve Opus 4.8 for genuinely ambiguous flows.

首先，查看项目根目录下的

.agents/qa-project-context.md

，跳过其中已回答的内容（技术栈、环境、种子/重置工具、模型权限）。

测试哪个流程，其UI多久变更一次？ 快速变更/实验性UI更适合意图驱动测试；稳定的关键路径（如登录）更适合脚本化测试。这将决定整体测试方案。
是否有种子测试数据和状态重置方式？ 没有种子数据和每次运行前的状态重置，就无法实现测试确定性。如果两者都不存在，这就是首要解决的问题。
能否绕过认证直接跳转到种子数据对应的入口？ 每次运行都重新执行登录是最大的可避免成本；使用种子数据对应的入口URL可以缩小测试目标范围并减少步骤。
明确的成功验证规则是什么？ 具体的账户文本、
```
/dashboard
```
URL、订单编号——加上禁止出现的状态。「无错误」不能作为验证规则。
目标应用是否基于Canvas/WebGL渲染？ 无无障碍树意味着优先快照的方式无法工作；需要规划视觉 fallback 方案，或为画布添加ARIA标记。
使用哪个模型以及预算多少？ 提前固定模型ID和步骤预算；普通导航步骤使用低成本模型（如Haiku 4.5 / Sonnet 4.6），仅在真正模糊的探索场景中使用Opus 4.8。

Core Principles

核心原则

Intent, not instructions — but only where churn earns it. The agent reads a goal and finds its own path through the accessibility tree, so it survives a moved button or renamed class that would break a selector. That resilience costs 2-5x the time and money of a scripted run, so spend it on fast-changing UI and hard-to-locate flows, not on stable critical paths.
An agent run is untrustworthy until it is deterministic. Same goal, same seeded app must produce the same verdict. That requires temperature 0, a pinned model id, seeded data with a reset, a bounded step budget, and an explicit pass/fail assertion. Without these you have a coin flip, not a test.
The oracle lives outside the agent. Never let the LLM self-grade "looks good." Success is a checkable assertion against the final
```
browser_snapshot
```
— specific expected text, a URL, AND a forbidden-state negative check — evaluated by your harness, not the model.
Accessibility tree first, pixels last.
```
browser_snapshot
```
returns roles, refs, and accessible names (~200-400 tokens) and is deterministic and cheap. Screenshots, pixel coordinates, vision, and OCR are a scoped last resort for canvas only, never the default.
Graduation is the goal, not perpetual agent runs. Once a flow is stable, promote the run to a durable scripted
```
tests/*.spec.ts
```
with role-based locators. An agent that has been green for two weeks should become a fast, free regression test — keep the agent for exploration, not for guarding a settled path.

关注意图而非指令——仅在UI频繁变更时使用。Agent读取目标并通过无障碍树自行寻找路径，因此可以承受按钮移动或类名修改这类会破坏选择器的变更。但这种韧性的代价是测试时间和成本是脚本化测试的2-5倍，因此应将其用于快速变更的UI和难以定位的流程，而非稳定的关键路径。
Agent测试运行具备确定性后才可信。相同的目标、相同的种子数据应用必须产生相同的测试结果。这需要将温度设为0、固定模型ID、带重置的种子数据、有限的步骤预算，以及明确的通过/失败断言。没有这些，测试就像掷硬币一样不可靠。
验证规则独立于Agent。绝不能让LLM自行判断「看起来没问题」。成功与否是通过最终的
```
browser_snapshot
```
进行可核查的断言——包括特定的预期文本、URL，以及禁止状态的反向检查——由测试框架而非模型评估。
优先使用无障碍树，像素作为最后手段。
```
browser_snapshot
```
返回角色、引用和无障碍名称（约200-400个token），具备确定性且成本低。截图、像素坐标、视觉模型和OCR仅作为画布应用的最后手段，绝不能作为默认方式。
最终目标是升级为脚本化测试，而非永久使用Agent。一旦流程稳定，就将Agent测试升级为持久化的脚本化
```
tests/*.spec.ts
```
测试，使用基于角色的定位器。连续两周保持绿色的Agent测试应转为快速、免费的回归测试——让Agent仅用于探索，而非守护已稳定的流程。

Fit: Intent-Driven vs Scripted

适配性：意图驱动 vs 脚本化

The decision is per-flow, not per-project. Run

risk-based-testing

first if you need the risk map; this table is the routing rule once you have it.

Flow characteristic	Use	Why
Stable, high-frequency critical path (login, payment)	Scripted + pinned ( `playwright-automation` )	Runs every PR; must be fast, free, and deterministic. No upside to re-exploring it.
Fast-changing / experimental UI (a dashboard that churns weekly, a redesign in flight)	Agentic / intent-driven	Selectors would break constantly; a goal survives layout churn.
Hard-to-locate flow you can't reliably select	Agentic	The agent finds the control by role/name instead of you reverse-engineering a selector.
Exploratory smoke / "does the happy path still work at all"	Agentic	One NL goal covers a lot of ground without a maintained script.
Anything in CI that must never falsely pass	Scripted, OR agentic with a hard oracle	Non-determinism is a false-pass risk you must actively cap.

The rule, stated plainly: keep stable critical paths scripted and pinned; point intent-driven agents at fast-changing UI and exploratory smoke. Do not move everything to the agent — it is slower, costlier, and non-deterministic, and not every test should be agentic.

该决策是针对单个流程而非整个项目的。如果需要风险地图，先执行

risk-based-testing

；有了风险地图后，以下表格可作为决策依据。

流程特征	使用方式	原因
稳定、高频的关键路径（登录、支付）	脚本化 + 固定配置（ `playwright-automation` ）	每个PR都会运行；必须快速、免费且具备确定性。重新探索该路径没有任何好处。
快速变更/实验性UI（每周更新的仪表盘、正在进行的重设计）	Agent化/意图驱动	选择器会频繁失效；基于目标的测试可以承受布局变更。
难以定位、无法可靠选择的流程	Agent化	Agent会通过角色/名称找到控件，无需你反向工程选择器。
探索性冒烟测试/「主流程是否仍能正常运行」	Agent化	一个自然语言目标可以覆盖大量场景，无需维护脚本。
CI中绝不能出现虚假通过的测试	脚本化，或带严格验证规则的Agent化	非确定性会带来虚假通过的风险，必须主动限制。

明确规则： 稳定的关键路径保持脚本化和固定配置；意图驱动的Agent用于快速变更的UI和探索性冒烟测试。不要将所有测试都转为Agent化——它更慢、成本更高且非确定性，并非所有测试都适合Agent化。

The Interaction Model (accessibility-tree-first)

交互模型（优先无障碍树）

Playwright MCP is not computer-use with screenshots and pixel coordinates. It is accessibility-tree-first:

```
browser_navigate
```
to the seeded entry URL.
```
browser_snapshot
```
returns the accessibility tree — each interactive element as a
```
role
```
, a stable
```
ref
```
, and its
```
accessible name
```
(from ARIA/labels). ~200-400 tokens.
The agent picks an element by
```
ref
```
and calls
```
browser_click
```
or
```
browser_type
```
.
```
browser_wait_for
```
waits on text appearing/disappearing — never a fixed sleep.
Re-
```
browser_snapshot
```
after the DOM changes; assert against that tree.

Why not screenshots: the snapshot is token-efficient (thousands of tokens cheaper than an image), deterministic (text refs, not fuzzy pixel matching), and needs no vision model or OCR. Feeding screenshots as the primary input makes the run slower, pricier, and flakier.

browser_take_screenshot

is for human evidence only, never as the assertion input.

See

references/setup.md

for the MCP registration, the full tool table, and the goal prompt.

Playwright MCP 不是使用截图和像素坐标的计算机操作。它优先基于无障碍树：

```
browser_navigate
```
跳转到种子数据对应的入口URL。
```
browser_snapshot
```
返回无障碍树——每个交互元素以
```
role
```
、稳定的
```
ref
```
和无障碍名称（来自ARIA/标签）的形式呈现。约200-400个token。
Agent通过
```
ref
```
选择元素，并调用
```
browser_click
```
或
```
browser_type
```
。
```
browser_wait_for
```
等待文本出现/消失——绝不要使用固定时长的睡眠。
DOM变更后重新调用
```
browser_snapshot
```
；基于该树进行断言。

为什么不使用截图：快照token效率更高（比图片便宜数千个token）、具备确定性（文本引用而非模糊像素匹配），且无需视觉模型或OCR。将截图作为主要输入会让测试运行更慢、成本更高且更不稳定。

browser_take_screenshot

仅用于人工查看证据，绝不能作为断言输入。

有关MCP注册、完整工具表和目标提示词，请查看

references/setup.md

。

Determinism: making a run trustworthy in CI

确定性：让测试在CI中可信

A run that passes once and fails the next with no app change is not yet a test. The fix is never "just retry" or bumping temperature for "smarter" exploration — that adds variance. Pin the variables instead:

Lever	Setting
Model	Pinned model id (e.g. `claude-haiku-4-5-20251001` ), never `latest`
Sampling	temperature 0 — no creative wandering in CI
Data	Seeded fixture + reset/seed the database before every run
Scope	Bounded step budget ( `maxSteps` ), e.g. 18 — exceeding it FAILS, never auto-retries
Oracle	Explicit pass/fail verdict asserted against the snapshot
Evidence	Assert on the accessibility tree, never a screenshot diff

Avoid:

temperature: 0.7

for exploration, retry-until-pass loops,

waitForTimeout

sleeps, and screenshot-based assertions. Each one hides flakiness rather than removing it. Full harness config in

references/setup.md

在应用未变更的情况下，一次通过、下次失败的运行不能称为测试。修复方案绝不是「重试」或提高温度以实现「更智能」的探索——这会增加方差。相反，应固定变量：

配置项	设置
模型	固定模型ID（例如 `claude-haiku-4-5-20251001` ），绝不要用 `latest`
采样	温度设为0——CI中不允许任何创造性探索
数据	种子测试数据 + 每次运行前重置/初始化数据库
范围	有限的步骤预算（ `maxSteps` ），例如18——超过预算则测试失败，绝不自动重试
验证规则	明确的通过/失败断言，基于快照验证
证据	基于无障碍树进行断言，绝不要用截图对比

避免：探索时使用

temperature: 0.7

或

、重试直到通过的循环、

waitForTimeout

睡眠，以及基于截图的断言。这些操作都会隐藏不稳定性而非消除它。完整的框架配置请查看

references/setup.md

。

Success Assertion: the Oracle (where agents fail silently)

成功断言：验证规则（Agent易静默失效的点）

This is the sharpest failure mode: the agent reports success while stuck on the login page, because "page loaded / no error / looks good" was accepted as success and the LLM was allowed to self-grade. Force an explicit oracle the harness checks — never the agent.

For the goal "sign in as an existing user and confirm the dashboard shows the right account name":

text

SUCCESS (all must hold — assert against the final browser_snapshot):
  - URL matches /dashboard
  - Snapshot contains the specific expected account name text, e.g. "Acme Corp — Jane R."
NEGATIVE / forbidden state (fail fast if any is true):
  - Still on a URL matching /login  → FAIL
  - Snapshot contains role="alert" with "invalid credentials"  → FAIL
VERDICT: harness emits {"passed": true|false}; the LLM does not decide.

The positive checks (specific account name +

/dashboard

URL) prove where it landed; the negative check (must NOT be on the login page) is what kills the false pass. "No error," "didn't crash," "screenshot looks correct," and "trust the agent" are not success criteria.

这是最严重的失效场景：Agent卡在登录页面却报告成功，因为「页面加载完成/无错误/看起来没问题」被当作成功标准，且允许LLM自行评估。必须强制使用由测试框架而非Agent检查的明确验证规则。

对于目标*「以现有用户身份登录，并确认仪表盘显示正确的账户名称」*：

text

SUCCESS（必须全部满足——基于最终的browser_snapshot断言）:
  - URL匹配/dashboard
  - 快照包含特定的预期账户名称文本，例如"Acme Corp — Jane R."
NEGATIVE / 禁止状态（任一为真则立即失败）:
  - 仍停留在匹配/login的URL → 失败
  - 快照包含role="alert"且文本为"invalid credentials" → 失败
VERDICT: 测试框架输出{"passed": true|false}; 不由LLM决定。

正向检查（特定账户名称 +

/dashboard

URL）证明Agent到达了正确位置；反向检查（绝不能停留在登录页面）是防止虚假成功的关键。「无错误」「未崩溃」「截图看起来正确」和「信任Agent」都不能作为成功标准。

Cost and Latency

成本与延迟

Agent runs are 2-5x slower and pricier than scripted tests — a step is an LLM round-trip, the dominant cost. Cut spend without losing coverage by going smaller, not bigger:

Step budget — keep
```
maxSteps
```
low and enforced; fewer round-trips, less drift.
Model tiering — Haiku 4.5 / Sonnet 4.6 for cheap navigation steps; reserve Opus 4.8 for genuinely ambiguous exploration. Don't run the biggest model on every step.
Prompt caching — cache the static system prompt, tool schemas, and goal; they repeat every run.
Scope via a seeded entry point — one narrow goal per run, deep-linked past login instead of re-driving it each time.
Snapshot over screenshots — the a11y snapshot is ~200-400 tokens; a full-page screenshot is thousands. Default to snapshot.

Backwards moves to reject: "use a bigger model / Opus 4.8 for every step," "raise the step limit," "screenshot every step," and running with no budget at all. See

references/setup.md

Agent测试的速度和成本是脚本化测试的2-5倍——每一步都是一次LLM往返，这是主要成本。通过缩小范围而非扩大范围来降低成本，同时不损失覆盖率：

步骤预算——保持
```
maxSteps
```
较低并严格执行；往返次数越少，偏差越小。
模型分层——普通导航步骤使用Haiku 4.5 / Sonnet 4.6；仅在真正模糊的探索场景中使用Opus 4.8。不要每一步都使用最大模型。
提示词缓存——缓存静态系统提示词、工具 schema 和目标；它们在每次运行中都会重复出现。
通过种子入口缩小范围——每次运行一个明确的小目标，直接跳转到登录后的页面，而非每次都重新执行登录。
优先使用快照而非截图——无障碍快照约200-400个token；全页截图则需要数千个token。默认使用快照。

拒绝反向操作：「每一步都使用更大的模型/Opus 4.8」「提高步骤限制」「每一步都截图」以及无预算限制的运行。请查看

references/setup.md

。

Graduation and CI Gating

升级路径与CI门禁

Promote a stabilized goal into a durable scripted test, and gate merges on the verdict. Both are detailed in

references/graduation-and-ci.md

; the essentials:

Graduate with Playwright Test Agents (planner / generator / healer, shipped in Playwright v1.56.0).
```
npx playwright init-agents --loop=claude
```
. The planner writes a Markdown test plan to
```
specs/<flow>.md
```
; the generator turns it into
```
tests/<flow>.spec.ts
```
with role-based locators (
```
getByRole
```
,
```
getByLabel
```
,
```
getByText
```
) verified against the live DOM; the healer repairs broken locators. This is the promotion path — not "keep running it as an agent," not recorded clicks, not
```
page.locator('xpath=...')
```
, not data-testid-only.
Gate CI so a failed goal exits non-zero and emits a machine-readable verdict (
```
{"passed": true|false}
```
in
```
result.json
```
); the GitHub Actions job parses the boolean and
```
exit 1
```
s on false. State is seeded/ephemeral and reset per run, with a step budget and a timeout cap. Never
```
continue-on-error: true
```
, never "always exit 0," never a prose verdict a human reads.
Canvas with no accessibility tree: prefer instrumenting the canvas with ARIA; as a scoped last resort enable
```
--caps=vision
```
to unlock
```
browser_mouse_click_xy
```
for that flow only.
```
browser_snapshot
```
will not work on a raw canvas, but don't make coordinates the default and don't abandon agentic testing.

将稳定的目标测试升级为持久化脚本化测试，并基于测试结果阻止代码合并。详细内容请查看

references/graduation-and-ci.md

；核心要点：

升级使用Playwright Test Agents（规划器/生成器/修复器，随Playwright v1.56.0发布）。执行
```
npx playwright init-agents --loop=claude
```
。规划器将测试计划写入
```
specs/<flow>.md
```
；生成器将其转换为
```
tests/<flow>.spec.ts
```
，使用基于角色的定位器（
```
getByRole
```
、
```
getByLabel
```
、
```
getByText
```
）并结合真实DOM验证；修复器会修复失效的定位器。这是正确的升级路径——不要「继续以Agent方式运行」，不要录制点击，不要使用
```
page.locator('xpath=...')
```
，不要仅依赖data-testid。
CI门禁：失败的测试目标应返回非零退出码并输出机器可读的结果（
```
result.json
```
中的
```
{"passed": true|false}
```
）；GitHub Actions任务解析该布尔值，若为false则
```
exit 1
```
。每次运行前都会初始化/重置临时状态，并强制执行步骤预算和超时限制。绝不要设置
```
continue-on-error: true
```
，绝不要「总是返回0」，绝不要输出需要人工阅读的文本结果。
无无障碍树的画布应用：优先为画布添加ARIA标记；仅作为最后手段，为该流程启用
```
--caps=vision
```
以解锁
```
browser_mouse_click_xy
```
。
```
browser_snapshot
```
在原生画布上无法工作，但不要将坐标作为默认方式，也不要放弃Agent化测试。

Migrating a brittle script to a goal (honest tradeoffs)

将脆弱的脚本转换为目标测试（坦诚的权衡）

Converting an 80-line script that re-types login and walks 6 hardcoded steps into a single NL goal with an explicit success assertion is a real win for a churning flow — but state the downsides honestly:

Non-determinism / false-pass risk — the run could pass falsely; that's why the hard oracle and the negative check are non-negotiable.
Cost/latency — 2-5x slower; bound it with a step budget and a seeded entry point.
Not every test should be agentic — keep stable paths scripted, and plan to graduate this one back to a scripted test once it stabilizes.

Reject the over-promise: it is not "strictly better with no downsides," do not "migrate everything," and never drop the assertions to make it pass.

将一个80行、重复执行登录并包含6个硬编码步骤的脚本转换为单个自然语言目标加明确的成功断言，对于频繁变更的流程来说是真正的胜利——但要坦诚说明缺点：

非确定性/虚假通过风险——测试运行可能会虚假通过；因此严格的验证规则和反向检查是必不可少的。
成本/延迟——速度慢2-5倍；通过步骤预算和种子入口来限制。
并非所有测试都应Agent化——稳定的路径保持脚本化，并计划在流程稳定后升级回脚本化测试。

拒绝过度承诺：它不是「绝对更好且无缺点」，不要「迁移所有测试」，绝不要为了让测试通过而放弃断言。

Anti-Patterns

反模式

1. Reflexively writing a scripted Playwright test

1. 习惯性编写脚本化Playwright测试

"Browser test" pattern-matches to codegen, so the default is

page.goto

page.locator

await expect(page...)

/ hunting

data-testid

. That misses the entire point. A goal-driven agent reads NL intent and explores via

browser_snapshot

— no pre-written selectors.

「浏览器测试」会让人联想到代码生成，因此默认选择

page.goto

page.locator

await expect(page...)

/寻找

data-testid

。这完全偏离了本技能的核心。基于目标的Agent读取自然语言意图并通过

browser_snapshot

探索——无需预编写选择器。

2. "Use the agent for everything"

2. 「所有测试都用Agent」

Over-selling the new toy. The agent is 2-5x slower and non-deterministic. Stable critical paths (login) stay scripted and pinned; intent-driven wins on fast-changing UI. Never "always use the agent" or "agents replace all scripted tests."

过度吹捧新工具。Agent测试速度慢2-5倍且非确定性。稳定的关键路径（如登录）保持脚本化和固定配置；意图驱动测试仅在快速变更的UI上有优势。绝不要「总是使用Agent」或「Agent替代所有脚本化测试」。

3. Fixing flakiness with retries or higher temperature

3. 用重试或提高温度修复不稳定性

"Just retry" and bumping temperature for "smarter" exploration both add variance. The real levers are temperature 0, a pinned model, seeded data, a bounded step budget, and an explicit verdict.

「重试」和提高温度以实现「更智能」的探索都会增加方差。真正有效的配置是温度设为0、固定模型、种子数据、有限步骤预算和明确的结果断言。

4. Assuming computer-use = screenshots + pixel coordinates

4. 认为计算机操作=截图+像素坐标

Playwright MCP is accessibility-tree-first. Defaulting to vision, screenshots, OCR, or

mouse_click_xy

is slower, costlier, and flakier. Pixels are a canvas-only last resort.

Playwright MCP优先基于无障碍树。默认使用视觉模型、截图、OCR或

mouse_click_xy

会更慢、成本更高且更不稳定。像素仅作为画布应用的最后手段。

5. "No error = success" (the false pass)

5. 「无错误=成功」（虚假通过）

Accepting "page loaded / didn't crash / looks good" and letting the LLM self-grade is exactly why the agent reports success while stuck on login. Require specific expected text, a URL, and a forbidden-state negative check, evaluated by the harness.

接受「页面加载完成/未崩溃/看起来没问题」并允许LLM自行评估，正是Agent卡在登录页面却报告成功的原因。要求特定的预期文本、URL，以及禁止状态的反向检查，由测试框架评估。

6. Bigger model / more steps to go faster

6. 使用更大模型/更多步骤来提高速度

Backwards. Opus 4.8 on every step and raising

maxSteps

raise cost and latency without buying reliability. Smaller models, tighter budgets, caching, and tighter scope are the fix.

这是反向操作。每一步都使用Opus 4.8和提高

maxSteps

会增加成本和延迟，却无法提升可靠性。正确的解决方案是使用更小的模型、更严格的预算、缓存和更窄的范围。

7. Running an agent forever instead of graduating

7. 永久运行Agent而不升级

A goal green for two weeks on a now-stable flow should become a scripted

tests/*.spec.ts

via Playwright Test Agents. "Keep running it as an agent," recording clicks, and xpath locators are all wrong promotions.

连续两周保持绿色的目标测试，且流程已稳定，应通过Playwright Test Agents转换为脚本化

tests/*.spec.ts

测试。「继续以Agent方式运行」、录制点击和xpath定位器都是错误的升级方式。

8. Prose verdict in CI

8. CI中返回文本结果

Returning a paragraph for a human to read, or

continue-on-error: true

, lets a failed goal merge. The run must exit non-zero on failure with a machine-readable boolean.

返回需要人工阅读的段落，或设置

continue-on-error: true

，会让失败的测试目标通过合并。测试运行失败时必须返回非零退出码，并输出机器可读的布尔值。

Done When

完成标准

Goal prompt exists as natural-language intent (no
```
page.locator
```
/
```
page.goto
```
/
```
data-testid
```
in the goal) with a START seeded entry URL.
An explicit success oracle is defined: specific expected text AND a URL check AND a forbidden-state negative check, asserted against
```
browser_snapshot
```
— not a screenshot.
Run config pins a model id, sets
```
temperature: 0
```
, a
```
maxSteps
```
budget, and a seed; no
```
waitForTimeout
```
, no retry-until-pass.

Interaction is snapshot-first:

browser_navigate

browser_snapshot

browser_click

browser_type

browser_wait_for

; screenshots used only for evidence.

The runner emits
```
result.json
```
with
```
{"passed": true|false}
```
and
```
exit 1
```
s on false; the CI job gates the merge on the boolean (no
```
continue-on-error
```
, no
```
always exit 0
```
).
CI seeds/resets ephemeral state per run and enforces a step budget and a timeout cap.
A graduation trigger is recorded (e.g. "green for 2 weeks → run
```
init-agents
```
, generate
```
tests/<flow>.spec.ts
```
with
```
getByRole
```
locators").
If any target is canvas/WebGL, the vision fallback (
```
--caps=vision
```
+
```
browser_mouse_click_xy
```
) is scoped to that flow only, or the canvas is instrumented with ARIA.

目标提示词为自然语言意图（目标中无
```
page.locator
```
/
```
page.goto
```
/
```
data-testid
```
），并包含初始种子入口URL。
定义了明确的成功验证规则：特定的预期文本 + URL检查 + 禁止状态的反向检查，基于
```
browser_snapshot
```
断言——而非截图。
运行配置固定了模型ID、设置
```
temperature: 0
```
、
```
maxSteps
```
预算和种子；无
```
waitForTimeout
```
，无重试直到通过的循环。

交互优先使用快照：

browser_navigate

browser_snapshot

browser_click

browser_type

browser_wait_for

；截图仅用于证据。

测试运行器输出
```
result.json
```
，包含
```
{"passed": true|false}
```
，失败时
```
exit 1
```
；CI任务基于该布尔值阻止合并（无
```
continue-on-error
```
，无
```
always exit 0
```
）。
CI每次运行都会初始化/重置临时状态，并强制执行步骤预算和超时限制。
记录了升级触发条件（例如「连续两周绿色 → 运行
```
init-agents
```
，生成带
```
getByRole
```
定位器的
```
tests/<flow>.spec.ts
```
」）。
如果目标是Canvas/WebGL应用，视觉fallback方案（
```
--caps=vision
```
+
```
browser_mouse_click_xy
```
）仅作用于该流程，或已为画布添加ARIA标记。