validate-e2e
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChineseValidate E2E
验证 E2E
Return evidence that the feature meets its acceptance contract: the finite requirements, observable behavior, failure cases, and runtime gates agreed for this run. Name the part of the contract that decided a result: "The contract requires the result to survive a reload, so a successful HTTP response is insufficient."
Keep progress in the ledger, not the transcript: the run's folder beside the feature's tasks () holds , , and . Resume from it. is in a source checkout and a sibling skill in a flat install.
validation/shared/references/work-state.mdplan.mdreport.mdevidence/shared/skills/shared/返回证据,证明该功能满足其验收契约:本次运行所约定的有限需求、可观察行为、失败场景和运行时门禁。指明契约中决定结果的具体条款:“契约要求结果在重新加载后仍然成立,因此仅返回成功的 HTTP 响应并不足够。”
进度应保留在 ledger 中,而不是 transcript 中:本次运行的 文件夹位于功能的 ()旁,其中包含 、 和 。应从该文件夹恢复工作。在源码检出目录中, 指 ;在扁平安装中,它指同级 skill。
validation/tasks/shared/references/work-state.mdplan.mdreport.mdevidence/shared/skills/shared/1. Bound the feature
1. 界定功能范围
Confirm the repository, worktree, intended base, current diff including uncommitted changes, and environment. Inventory only changed surfaces, their direct callers, and named external interfaces. Record each requirement, operation, relevant variants, expected result, and the check and browser observation IDs that prove it. Keep the denominator finite: say whether "100%" means requirements, a closed operation inventory, or measured line or branch coverage with named thresholds. Timebox discovery; a missing source or policy becomes a named blocker, not a wider search.
Keep behavior, browser, security, migration, performance, and runtime gates separate. A source finding is not execution evidence, and a business pass never overrides a failed runtime gate.
Choose the mode. reports defects without touching product code. fixes them within named paths and needs a request to fix defects. Invocation already permits local test preparation, execution, and disposable fixtures; production writes, external messages, deployments, commits, and publication need their own authority.
assessrepairWrite after the inventory:
plan.md- the mode, scope, base, requirement IDs, and whether discovery is closed (an open inventory blocks an overall pass);
- the units: each with its requirements, whether it is required, its check and browser observation IDs, and its attempt limit;
- the roles, if any, and the ceilings: total attempts, role calls per route and in total, parallel workers, and a deadline. Propose 12 attempts, 12 role calls, 3 per role, 2 per unit, 2 workers, and 90 minutes as editable starting values. Repair allowances sit inside those totals.
Every requirement maps to a required unit. The ceilings bound what you start; they do not stop a running call, so they are not a spend cap.
确认仓库、工作树、预期基线、当前 diff(包括未提交的更改)和环境。仅盘点发生变更的界面、它们的直接调用方以及明确指定的外部接口。记录每项需求、操作、相关变体、预期结果,以及用于证明它们的检查和浏览器观察 ID。保持分母有限:明确“100%”是指需求比例、封闭的操作清单,还是按指定阈值测量的行覆盖率或分支覆盖率。对发现工作设置时间限制;缺少源码或策略时,应将其记录为明确的阻塞项,而不是扩大搜索范围。
分别处理行为、浏览器、安全、迁移、性能和运行时门禁。源码层面的发现不属于执行证据;业务检查通过也绝不能覆盖失败的运行时门禁。
选择模式。 仅报告缺陷,不修改产品代码。 会在指定路径内修复缺陷,且需要明确提出修复请求。调用过程本身已允许进行本地测试准备、执行和使用一次性 fixture;生产环境写入、外部消息、部署、提交和发布则需要各自的授权。
assessrepair完成盘点后编写 :
plan.md- 模式、范围、基线、需求 ID,以及发现工作是否已封闭(未封闭的清单会阻止整体判定为通过);
- 各单元:每个单元包含其需求、是否为必需单元、检查和浏览器观察 ID,以及尝试次数上限;
- 角色(如有)及各项上限:总尝试次数、各路由和总计的角色调用次数、并行 worker 数量和截止时间。建议以 12 次尝试、12 次角色调用、每个角色 3 次、每个单元 2 次、2 个 worker 和 90 分钟作为可编辑的初始值。修复 allowance 应包含在这些总计之内。
每项需求都必须映射到一个必需单元。这些上限用于约束任务的启动规模;它们不会中止已在运行的调用,因此不等同于支出上限。
2. Route workers only where needed
2. 仅在需要时为 worker 分配路由
Build, lint, type checks, and existing suites run through repository commands with no model. A model is for a decision: missing tests, unsettled expected behavior, an ambiguous browser state, or an unexplained failure. When the work is command-only, the plan names no roles.
For a worker, resolve the route that fits with :
python3 shared/scripts/model_routing.py resolve <route> --provider <host> [--risk high]| Work | Route |
|---|---|
| Scope, requirement mapping, final evidence assessment | |
| Missing unit, component, or property tests from agreed cases | |
| API, database, queue, migration, integration, performance diagnosis | |
| Interactive browser journeys, visual state, accessibility | |
| Authorization, money, concurrency, data loss, complex root cause | |
| Diagnosis and correction from a proven cause | |
| Independent assessment of evidence and missing cases | |
A strong role settles expected behavior before a cheaper role writes the tests. A browser worker needs real browser tools in its own process. The reviewer's model differs from any model that wrote the assertions.
Show the scope, each role with its model, effort, and host route, and the attempt, call, and time ceilings. Ask: "Approve this plan, or specify changes." A request to use defaults and run approves the unchanged resolved plan. Then route each role by host: T3 Code (continue with ; a Muse route uses ); Claude Code or Codex a native subagent; otherwise , , or . Give each role one persistent context for the run. An unavailable route blocks its own work; never swap in another model.
delegate_taskt3_thread_sendmuse-runnercli-runnerpi-runnermuse-runner构建、lint、类型检查和现有测试套件均通过仓库命令运行,无需模型。只有需要作出决策时才使用模型,例如缺少测试、预期行为尚未确定、浏览器状态存在歧义或出现无法解释的失败。若工作仅涉及命令执行,计划中不设置任何角色。
为 worker 分配任务时,使用 解析适合的路由:
python3 shared/scripts/model_routing.py resolve <route> --provider <host> [--risk high]| 工作 | 路由 |
|---|---|
| 范围界定、需求映射、最终证据评估 | |
| 根据已约定的场景补充单元、组件或属性测试 | |
| API、数据库、队列、迁移、集成或性能问题诊断 | |
| 交互式浏览器流程、视觉状态、无障碍 | |
| 授权、金额、并发、数据丢失或复杂根因 | |
| 根据已证实的原因进行诊断和修正 | |
| 独立评估证据和遗漏场景 | |
应先由能力更强的角色确定预期行为,再由成本较低的角色编写测试。浏览器 worker 需要在其自身进程中运行真实的浏览器工具。审查者使用的模型必须与编写断言的模型不同。
展示范围、每个角色及其模型、工作量和主机路由,以及尝试次数、调用次数和时间上限。询问:“批准此计划,或指定需要更改的内容。” 请求使用默认值并运行,即视为批准未作更改的已解析计划。随后根据主机为每个角色分配路由:T3 Code 使用 (通过 继续;Muse 路由使用 );Claude Code 或 Codex 使用原生 subagent;其他情况使用 、 或 。为每个角色提供本次运行的持久上下文。某个路由不可用时,仅阻塞该角色负责的工作;绝不替换为其他模型。
delegate_taskt3_thread_sendmuse-runnercli-runnerpi-runnermuse-runner3. Run the smallest sufficient checks
3. 运行最小充分检查
Read only the applicable rows of references/test-methods.md. Reuse evidence only when requirements, inputs, dependency versions, environment, and check definitions all match; a commit ID change alone does not invalidate it. A declared fresh check always runs.
Write each attempt into before running it; a failed or interrupted attempt still counts. Run cheap relevant checks first, then the real integration and user flows the contract needs. Capture every command with into . A browser observation records its ID, result, and screenshot and trace paths; check browser access first and record it with ; a blocked preflight consumes no attempt. A unit passes only when each of its checks has a current capture with the expected result. A prose pass does not execute a command or pass a unit.
report.mdshared/scripts/capture_check.py runevidence/shared/scripts/browser_preflight.py recordParallelize only independent work with isolated fixtures, accounts, ports, and browser contexts; one coordinator writes state. Keep full logs in files and read counts, failures, and paths.
仅阅读 references/test-methods.md 中适用的行。只有当需求、输入、依赖版本、环境和检查定义全部一致时,才能复用证据;仅更改 commit ID 并不会使证据失效。声明为重新执行的检查始终需要运行。
每次尝试都必须在执行前写入 ;失败或中断的尝试仍计入次数。使用 运行每个命令,并将结果捕获到 中。浏览器观察应记录其 ID、结果、截图和 trace 路径;先使用 检查浏览器访问能力并记录结果;预检被阻塞时不消耗尝试次数。只有当单元的每项检查都有当前捕获且结果符合预期时,该单元才算通过。纯文字形式的通过既不执行命令,也不能使单元通过。
report.mdshared/scripts/capture_check.py runevidence/shared/scripts/browser_preflight.py record仅在任务相互独立且使用隔离的 fixture、账户、端口和浏览器上下文时并行执行;由一个协调者统一写入状态。完整日志保存在文件中,读取时仅提取计数、失败信息和路径。
4. Diagnose and repair within bounds
4. 在限定范围内诊断和修复
Classify each failure as product behavior, test harness, environment, runtime gate, or unresolved policy, with actual and expected values. Cascading failures from one setup error are one cause until evidence shows more. Use for an unknown cause, and never rerun an unchanged failure without a new hypothesis.
diagnoseAllow two validation attempts per unit by default. Test, evidence, and product repair allowances are set in the plan and share its total attempt, call, and time limits; each retry records its reason. At a ceiling, stop and report the remaining work. Never start a new run, rename a unit, or weaken an assertion, exclusion, timeout, or diagnostic policy to reset counts or obtain a pass. A stronger model can resolve uncertainty; it cannot choose an unsettled product policy.
将每次失败归类为产品行为、测试 harness、环境、运行时门禁或未解决策略,并记录实际值和预期值。对于由同一设置错误引发的级联失败,在证据表明存在其他原因之前,应视为同一个原因。未知原因使用 ;没有新假设时,绝不重复运行未作更改的失败检查。
diagnose默认允许每个单元进行两次验证尝试。测试、证据和产品修复 allowance 由计划设定,并共享其总尝试次数、调用次数和时间限制;每次重试都要记录原因。达到上限时,停止执行并报告剩余工作。绝不启动新运行、重命名单元,或通过削弱断言、排除条件、超时设置或诊断策略来重置计数或取得通过。更强的模型可以解决不确定性,但不能替尚未确定的产品策略作出选择。
5. Close or checkpoint
5. 结束或保存进度
On resume, find out what pending calls and attempts did before any retry. requires a closed inventory, every required unit passed with current captures of one tree, and no required failure, skip, pending call, or blocker. Otherwise report , , , or . Captures prove facts, not their meaning: get an independent review of the same tree before claiming readiness. Never describe a scoped pass as universal coverage.
passedfailedblockedpartialceiling_hitReturn:
- Status, requirement and operation scope, tested revision and environment.
- Required units passed out of total; failed, blocked, skipped, pending, and excluded units with reasons. Code coverage only from its actual report.
- Checks run now, reused evidence, defects found, repairs made, and open policy decisions.
- Each role's model, effort, and route, with call counts and usage when available; unknown stays unknown.
- Plan, report, and evidence paths, the next bounded action, and remaining budget if incomplete.
恢复运行时,在任何重试之前先查明待处理调用和尝试此前完成的工作。判定为 必须满足:清单已封闭、所有必需单元均通过、同一棵代码树具有当前捕获证据,并且不存在必需失败、跳过、待处理调用或阻塞项。否则应报告 、、 或 。捕获证据只能证明事实,不能证明其含义;在声明可以交付前,应由独立审查者审查同一棵代码树。绝不将限定范围内的通过描述为普遍覆盖。
passedfailedblockedpartialceiling_hit返回:
- 状态、需求和操作范围、所测试的 revision 及环境。
- 已通过的必需单元数及单元总数;失败、被阻塞、跳过、待处理和排除的单元及其原因。代码覆盖率只能引用实际生成的报告。
- 本次运行的检查、复用的证据、发现的缺陷、实施的修复,以及尚未解决策略决策。
- 每个角色使用的模型、工作量和路由,并在可用时提供调用次数和用量;未知信息保持为未知。
- 计划、报告和证据路径,下一个有界操作,以及未完成时的剩余预算。