selector-drift-recovery

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese
<objective> A redesign shipped and 23 tests now fail with `TimeoutError: locator.* exceeded` — the DOM moved, not the product. This skill closes that maintenance loop: it diffs the old DOM against the new one, regenerates the broken selectors role-first against the new build, validates the whole suite, and ships the diff as one reviewable PR with a confidence score and screenshot per change. The trigger is an event (a refactor merged), not a flake. The output is a PR a human signs off on — not a silent runtime auto-heal.
This is the bulk, offline, batch counterpart to
test-reliability
. They share the multi-attribute locator + confidence scoring primitives but run in opposite directions:
test-reliability
heals one selector at runtime behind a guarded threshold;
selector-drift-recovery
regenerates N selectors offline against the new DOM and bundles them into a PR. If you are doing the second workflow inside
test-reliability
, switch here. </objective>
<objective> 重新设计上线后,有23个测试因`TimeoutError: locator.* exceeded`失败——问题出在DOM结构变更,而非产品功能。本技能可完成这一维护闭环:对比新旧DOM结构,针对新版本优先按角色重新生成失效选择器,验证整个测试套件,并将差异打包为一个可评审的PR,每项变更附带置信度评分和截图。触发条件是事件(重构已合并),而非测试不稳定。输出结果是需要人工确认的PR,而非静默的运行时自动修复。
这是
test-reliability
的批量离线版本。二者共享多属性定位器+置信度评分核心逻辑,但运行方向相反:
test-reliability
在运行时针对单个选择器进行阈值管控下的修复;
selector-drift-recovery
则离线针对新DOM重新生成N个选择器并打包为PR。如果你在
test-reliability
中执行第二种工作流,请切换到本技能。 </objective>

Quick Route

快速指引

SituationGo to
One broken test, not a refactorStop — use
test-reliability
instead
200+ broken selectors across many filesSplit by area first (one PR per page/dir), then Phase 1
Refactor changed flows/semantics, not just structureStop — rewrite from specs with
playwright-automation
Framework switch (Selenium → Playwright)Stop — use
test-migration
, not drift recovery
No old-DOM reference exists anywhereCapture one (Phase 1) or scope down — without it this is "rewrite tests"
Have old + new DOM, ready to mapPhase 1 → 6 below

场景对应技能
单个测试失效,非重构导致停止——改用
test-reliability
跨多个文件存在200+个失效选择器先按区域拆分(每个页面/目录一个PR),再执行第一阶段
重构变更了流程/语义,而非仅结构停止——使用
playwright-automation
根据需求重写测试
框架切换(Selenium → Playwright)停止——使用
test-migration
,而非漂移修复
无旧DOM参考先捕获旧DOM(第一阶段)或缩小范围——无旧DOM时等同于「重写测试」
已有新旧DOM,准备映射执行以下第一至第六阶段

Discovery Questions

调研问题

Check
.agents/qa-project-context.md
first — if it exists, use it and skip anything answered there. It identifies your E2E framework, selector strategy, and known fragile areas. Then:
  1. What triggered the drift? A planned refactor (Storybook can show the new DOM before merge), a shipped redesign (new DOM is in main), a dependency upgrade, or a Tailwind/CSS migration? The trigger decides whether you run pre-emptively or react to CI failures.
  2. What is the blast radius? A single component, a page, or the whole app? Single component: scope recovery to the test files that touch it. Global: budget half a day to a day, and decide which tests should be rewritten rather than re-selected.
  3. What is your current selector strategy? If selectors are mostly
    data-testid
    and the refactor preserved testids, recovery is trivial. If they are CSS-class or XPath based, expect 30–60% to need a new strategy, not just a new locator.
  4. Is there a passing baseline? You need the old DOM somewhere: a previous CI trace artifact, a deployed staging build, a Storybook story, or git history of the components. No old-DOM reference means this degrades to "rewrite tests."
  5. Are the broken locators inline or wrapped in a Page Object? The JSON reporter's
    error.location
    points at the failing line. For inline locators that is the locator itself; for POM-wrapped locators it points at the POM helper, not the test. Know this before you trust the auto-extracted line numbers (see Failure Modes).
  6. Who reviews the resulting PR? Confidence-scored updates need a human signoff. Decide upfront whether the PR goes to the test author, the engineer who did the refactor, or the QA lead.

首先查看
.agents/qa-project-context.md
——如果存在,使用其中信息并跳过已回答的问题。该文件会指明你的E2E框架、选择器策略及已知脆弱区域。然后:
  1. 是什么触发了漂移? 计划内重构(Storybook可在合并前展示新DOM)、已上线的重新设计(新DOM在主分支)、依赖升级,还是Tailwind/CSS迁移?触发方式决定了你是提前执行还是响应CI失败。
  2. 影响范围多大? 单个组件、单个页面,还是整个应用?单个组件:将修复范围限定在涉及该组件的测试文件;全局范围:预留半天到一天时间,并决定哪些测试应重写而非重新选择。
  3. 当前使用的选择器策略是什么? 如果选择器大多是
    data-testid
    且重构保留了testid,修复会非常简单;如果是基于CSS类或XPath的选择器,预计30%-60%需要更换策略,而非仅更换定位器。
  4. 是否有可用的通过基线? 你需要在某处获取旧DOM:之前的CI跟踪 artifact、已部署的预发布版本、Storybook示例,或组件的git历史。无旧DOM参考时,本技能退化为「重写测试」。
  5. 失效的定位器是内联还是封装在Page Object中? JSON报告的
    error.location
    指向失败行。对于内联定位器,该行就是定位器本身;对于POM封装的定位器,该行指向POM助手而非测试代码。在信任自动提取的行号前需明确这一点(参见故障模式)。
  6. 谁来评审最终PR? 带置信度评分的更新需要人工确认。提前确定PR提交给测试作者、执行重构的工程师还是QA负责人。

Core Principles

核心原则

  1. Recovery is event-driven, not failure-driven. Run this when a refactor is planned or just shipped, not when one test goes flaky. One broken test →
    test-reliability
    . Ten or more from the same event → this skill.
  2. Old DOM, new DOM, mapped pair. The whole skill rests on a snapshot before the refactor and one after. Everything else is bookkeeping around that pair. Capture both as aria snapshots (
    await page.locator('body').ariaSnapshot()
    ), not raw HTML — a role-tree diff is exactly the signal role-first recovery needs and ignores the cosmetic churn (class renames, wrapper divs) that a raw-HTML diff drowns in. If you cannot produce both snapshots, fix that first.
  3. Role-first replacement, every time. Even when the old test used CSS, the regenerated selector should prefer
    getByRole
    + accessible name, then
    getByLabel
    for form fields, then
    getByTestId
    when the refactor added one. The recovery PR is your chance to ratchet the average selector stability score up (0–5 rubric below, shared with
    test-reliability
    ).
  4. Disambiguate by region scoping, not layout selectors. When role+name is ambiguous (two "Submit" buttons), narrow with
    getByRole('region', { name }).getByRole('button', …)
    or
    getByRole(...).filter({ hasText })
    . Do not reach for
    :near()
    /
    :right-of()
    — see the Avoid note. A score-3 candidate is one where region scoping has been applied and the locator now matches exactly one element.
  5. One PR, grouped by file, with per-change evidence. Reviewers cannot eyeball 47 selector changes spread across 30 commits. Bundle into one PR, group hunks by test file, attach a confidence score + DOM screenshot per change.
  6. The suite must pass before merge, and dead tests get deleted. A regenerated selector that doesn't run is a worse version of the original problem; the skill ends with green CI, not a generated diff. And if the refactor removed a feature, prune its tests — do not regenerate selectors for elements that no longer exist.
Avoid: Playwright layout selectors
:near()
,
:right-of()
,
:left-of()
,
:above()
,
:below()
as disambiguators — officially deprecated and "may be removed," because a 1px layout shift changes the match (Playwright docs, 2026). They also contradict the role-first thesis. Use region scoping /
getByRole().filter()
instead.

  1. 修复由事件驱动,而非失败驱动。 在计划重构或重构刚完成时执行本技能,而非单个测试不稳定时。单个失效测试→使用
    test-reliability
    ;同一事件导致10个及以上测试失效→使用本技能。
  2. 旧DOM、新DOM、映射配对。 本技能的核心依赖重构前后的两个快照。所有其他操作都是围绕这对快照的辅助工作。将两者捕获为aria快照
    await page.locator('body').ariaSnapshot()
    ),而非原始HTML——角色树对比正是优先按角色修复所需的信号,且会忽略原始HTML对比中会干扰判断的 cosmetic 变更(类名重命名、包装div等)。如果无法生成两个快照,请先解决此问题。
  3. 始终优先按角色替换。 即使旧测试使用CSS选择器,重新生成的选择器也应优先使用
    getByRole
    +可访问名称,然后是表单字段的
    getByLabel
    ,最后是重构新增的
    getByTestId
    。修复PR是提升选择器平均稳定性评分的机会(评分标准为0-5,与
    test-reliability
    共享)。
  4. 通过区域范围消除歧义,而非布局选择器。 当角色+名称存在歧义时(如两个“Submit”按钮),使用
    getByRole('region', { name }).getByRole('button', …)
    getByRole(...).filter({ hasText })
    缩小范围。请勿使用
    :near()
    /
    :right-of()
    ——参见注意事项。评分3的候选选择器是指已应用区域范围且定位器仅匹配一个元素的情况。
  5. 单个PR,按文件分组,附带每项变更证据。 评审人员无法逐一查看分散在30个提交中的47项选择器变更。将所有变更打包为一个PR,按测试文件分组,每项变更附带置信度评分+DOM截图。
  6. 合并前套件必须通过测试,失效测试需删除。 无法运行的重新生成选择器比原问题更糟;本技能的终点是CI通过,而非生成差异。如果重构移除了某个功能,需删除对应测试——不要为已不存在的元素重新生成选择器。
注意事项: 避免使用Playwright布局选择器
:near()
:right-of()
:left-of()
:above()
:below()
作为歧义消除方式——官方已标记为弃用,“可能会被移除”,因为1px的布局偏移就会改变匹配结果(Playwright文档,2026)。同时这也与优先按角色的原则相悖,请改用区域范围划分/
getByRole().filter()

Workflow

工作流程

Six phases, each gated by a check before the next.
分为六个阶段,每个阶段完成后需通过检查才能进入下一阶段。

Phase 1 — Snapshot the old DOM

阶段1 — 快照旧DOM

You need a snapshot of every page or component the affected tests touch, in its pre-refactor state. Sources, in preference order:
  1. Last green CI trace artifact. Most teams save Playwright traces on failure (
    trace: 'on-first-retry'
    ). Download the green-run artifact, open a trace with
    npx playwright show-trace traces/checkout.zip
    , select an action, and read the per-action DOM snapshot panel for each surface. (The viewer no longer has a "Copy HTML at this step" menu item; you read the snapshot panel or, for a programmatic dump, replay with
    page.content()
    /
    ariaSnapshot()
    .)
  2. Storybook at a pre-refactor commit.
    git checkout <PRE_REFACTOR_SHA>
    , start Storybook, and dump each story with a tiny
    page.content()
    /
    ariaSnapshot()
    script.
  3. A staging build still on the old version. Navigate the same flows and snapshot.
  4. Git history of the components. Reconstructable but the most expensive — render in isolation.
Output:
.drift-recovery/old/<page-or-component>.aria.yml
(and
.html
if you also need raw markup) per affected unit.
Gate: You can answer "what did this page look like when the tests last passed?" from a snapshot file, not from memory.
你需要获取受影响测试涉及的每个页面或组件在重构前的状态快照。来源优先级:
  1. 最后一次通过的CI跟踪artifact。 大多数团队会在测试失败时保存Playwright跟踪(
    trace: 'on-first-retry'
    )。下载通过版本的artifact,使用
    npx playwright show-trace traces/checkout.zip
    打开跟踪记录,选择操作步骤,查看每个界面的DOM快照面板。(查看器不再有“复制此步骤的HTML”菜单项;你可以读取快照面板,或通过
    page.content()
    /
    ariaSnapshot()
    以编程方式导出。)
  2. 重构前提交的Storybook。 执行
    git checkout <PRE_REFACTOR_SHA>
    ,启动Storybook,使用小型
    page.content()
    /
    ariaSnapshot()
    脚本导出每个示例。
  3. 仍为旧版本的预发布构建。 导航相同流程并生成快照。
  4. 组件的git历史。 可重构但成本最高——需单独渲染。
输出:每个受影响单元对应
.drift-recovery/old/<page-or-component>.aria.yml
(若需原始标记可同时生成
.html
文件)。
检查点: 你可以通过快照文件而非记忆回答“测试最后通过时该页面是什么样子?”

Phase 2 — Snapshot the new DOM

阶段2 — 快照新DOM

Run the same surfaces in the post-refactor build — a Vercel/Netlify preview deploy is ideal, or a local dev server / the PR branch in CI. Wait for hydration (
await page.waitForLoadState('networkidle')
) before snapshotting, or SSR pages give you the pre-hydration tree and you miss client-rendered elements.
Output:
.drift-recovery/new/<page-or-component>.aria.yml
matching the old set.
Gate: Every old snapshot has a matching new one. If a route 404s in the new build, that flow was deleted — mark its tests for the deletion pile in Phase 6.
在重构后的版本中运行相同界面——Vercel/Netlify预览部署是理想选择,也可使用本地开发服务器/CI中的PR分支。在生成快照前等待 hydration 完成(
await page.waitForLoadState('networkidle')
),否则SSR页面会返回预hydration树,导致缺失客户端渲染元素。
输出:与旧快照对应的
.drift-recovery/new/<page-or-component>.aria.yml
文件。
检查点: 每个旧快照都有对应的新快照。如果新构建中某路由返回404,说明该流程已被删除——在阶段6标记对应测试为待删除。

Phase 3 — Identify broken selectors and infer intent

阶段3 — 识别失效选择器并推断意图

For each test file, run against the new build with the JSON reporter, then parse it. Capture, per failure: file, line, old locator string, error type (timeout vs assertion), and inferred intent. Group the results by test file.
  • Error classification: a drift failure is
    TimeoutError: locator.* exceeded
    . Distinguish it from an assertion failure (
    expect(...).toBe
    ) so you don't try to re-select a locator that resolved fine but failed a value check.
  • Inferred intent is mandatory and not in the reporter. Read the surrounding test code — what action is taken on the locator, what assertion follows — and store a short intent string ("submit the order", "read the order total"). The candidate generator keys off intent, so this step is load-bearing, not commentary.
  • Page route is also not in the reporter. Map each locator to the snapshot it should resolve against (which
    .drift-recovery/new/*.aria.yml
    ) so Phase 4 can load the right new DOM.
The result is a per-file table:
Test fileLineOld locatorError typePage routeInferred intent
tests/checkout.spec.ts
42
getByTestId('submit-btn')
timeout
/checkout
Submit the order
tests/checkout.spec.ts
87
locator('.summary > h2')
timeout
/checkout
Read the order total
See
references/recovery-scripts.md
for
identify-drift.ts
, which produces exactly these rows (with the intent/route fields populated, not stubbed).
Gate: Every broken locator has an inferred intent and a page route. If you cannot infer intent, ask the test author or read the original PR — do not guess.
针对每个测试文件,在新版本上运行并使用JSON报告器,然后解析报告。捕获每项失败的文件、行号、旧定位器字符串、错误类型(超时vs断言)及推断意图。按测试文件分组结果。
  • 错误分类: 漂移失败为
    TimeoutError: locator.* exceeded
    。需与断言失败(
    expect(...).toBe
    )区分,避免尝试重新选择已正常解析但值检查失败的定位器。
  • 推断意图是必需的,且报告中不包含该信息。 阅读测试代码上下文——定位器执行的操作是什么,后续的断言是什么——并存储简短的意图字符串(如“提交订单”、“读取订单总额”)。候选选择器生成器依赖此意图,因此这一步是核心,而非注释。
  • 页面路由也不在报告中。 将每个定位器映射到其应解析的快照(即
    .drift-recovery/new/*.aria.yml
    ),以便阶段4加载正确的新DOM。
结果为按文件分组的表格:
测试文件行号旧定位器错误类型页面路由推断意图
tests/checkout.spec.ts
42
getByTestId('submit-btn')
超时
/checkout
提交订单
tests/checkout.spec.ts
87
locator('.summary > h2')
超时
/checkout
读取订单总额
参见
references/recovery-scripts.md
中的
identify-drift.ts
,该脚本可生成上述完整行(已填充意图/路由字段,而非占位符)。
检查点: 每个失效定位器都有推断意图和页面路由。如果无法推断意图,请询问测试作者或查看原始PR——请勿猜测。

Phase 4 — Generate replacement candidates

阶段4 — 生成替换候选选择器

For each row, generate candidates against the new DOM snapshot and score each on the 0–5 rubric (shared with
test-reliability
). Strategy ladder, best first:
  1. New
    data-testid
    added by the refactor team — the most stable choice they made. Score 5.
  2. getByRole
    + accessible name, unique on the page.
    Score 4.
  3. getByLabel
    for a form field
    , when the intent is an input and a label exists. Score 4 (use over a bare role when the field has no name otherwise).
  4. Role + name, region-scoped to a single match. If role+name alone returns >1 element, wrap it —
    getByRole('region', { name }).getByRole(role, { name })
    or
    .filter({ hasText })
    — and confirm the scoped locator now matches exactly one. Only score 3 after scoping makes it unambiguous.
  5. Visible text only (
    getByText
    ). Score 2 — fragile to copy changes.
  6. CSS class on the changed structure. Score 1 — usually still broken.
  7. No safe replacement. Score 0 — flag for human.
ScoreReplacement strategyAuto-apply?
5New
data-testid
exists
yes
4
getByRole
+ accessible name (or
getByLabel
), unique on page
yes
3
getByRole
+ name, region-scoped to exactly one match
yes
2Visible-text-onlyno
1CSS class on changed structureno
0No safe replacement foundno — flag for human
A candidate is score 3 only if scoping already resolved it to a single element. A still-ambiguous multi-match (
count > 1
, "needs scoping") is not a 3 — it is unfinished, and must not be auto-applied.
Output:
.drift-recovery/candidates.json
with
{ file, line, oldLocator, selector, score, rationale, screenshotPath }
per change. See
references/recovery-scripts.md
for
generate-candidates.ts
.
Gate: Every row has a candidate scored ≥ 3 (and confirmed unique), or is flagged for human review. Score-0/1/2 are never auto-applied.
针对每一行,在DOM快照上生成候选选择器,并按0-5评分标准(与
test-reliability
共享)评分。策略优先级从高到低:
  1. 重构团队新增的
    data-testid
    ——最稳定的选择。评分5。
  2. getByRole
    +可访问名称,在页面上唯一。
    评分4。
  3. 表单字段的
    getByLabel
    ——当意图是输入且存在标签时使用。评分4(当字段无其他名称时,优先于单纯的角色选择器)。
  4. 角色+名称,通过区域范围限定为单一匹配。 如果仅角色+名称返回>1个元素,需进行包装——
    getByRole('region', { name }).getByRole(role, { name })
    .filter({ hasText })
    ——并确认范围限定后的定位器仅匹配一个元素。仅在范围限定消除歧义后才可评为3分。
  5. 仅可见文本
    getByText
    )。评分2——易受文本变更影响。
  6. 变更结构上的CSS类。 评分1——通常仍会失效。
  7. 无安全替换方案。 评分0——标记为需人工处理。
评分替换策略自动应用?
5存在新增的
data-testid
4
getByRole
+可访问名称(或
getByLabel
),在页面上唯一
3
getByRole
+名称,通过区域范围限定为单一匹配
2仅可见文本
1变更结构上的CSS类
0未找到安全替换方案否——标记为需人工评审
候选选择器仅在范围限定后解析为单一元素时才可评为3分。仍存在歧义的多匹配(
count > 1
,“需范围限定”)不能评为3分——属于未完成状态,不得自动应用。
输出:
.drift-recovery/candidates.json
,每项变更包含
{ file, line, oldLocator, selector, score, rationale, screenshotPath }
。参见
references/recovery-scripts.md
中的
generate-candidates.ts
检查点: 每一行都有评分≥3的候选选择器(且已确认唯一),或已标记为需人工评审。评分0/1/2的候选选择器绝不自动应用。

Phase 5 — Apply, validate, iterate

阶段5 — 应用、验证、迭代

  1. Apply the score-≥3 replacements to a feature branch. Replace by
    (file, line)
    , not a content-wide string replace — the reporter's locator string is a rendered form (
    locator('.summary > h2')
    ) that rarely matches the source expression verbatim, and a naive
    String.replace
    hits only the first occurrence and collides on identical locators. Edit the specific line; set
    applied: true
    on each candidate you actually wrote.
  2. Run the full affected suite, not just the previously-failing tests — a new selector can match an unintended element and break a previously-passing test.
  3. Per test: passed → keep the replacement. failed → revert that one line, mark the test for human review.
  4. Emit a summary: N recovered automatically, M flagged.
Gate: Recovered tests pass the suite. Flagged tests are clearly marked, not silently included.
  1. 将评分≥3的替换项应用到功能分支。按
    (file, line)
    进行替换,而非全局字符串替换——报告中的定位器字符串是渲染后的形式(如
    locator('.summary > h2')
    ),很少与源代码表达式完全匹配,简单的
    String.replace
    只会命中第一个匹配项,且会在相同定位器上发生冲突。编辑指定行;在实际应用的候选选择器上标记
    applied: true
  2. 运行所有受影响的测试套件,而非仅之前失败的测试——新选择器可能匹配非预期元素,导致之前通过的测试失效。
  3. 针对每个测试:通过→保留替换项;失败→还原该行,标记测试为需人工评审。
  4. 生成总结:自动修复N个,标记M个需人工处理。
检查点: 修复后的测试通过套件验证。标记的测试需明确标识,而非静默包含。

Phase 6 — Ship the PR

阶段6 — 提交PR

The PR is the deliverable. Title:
chore(tests): selector recovery after <refactor description>
. Body (generated from
candidates.json
, filtering on
applied
):
markdown
undefined
PR是最终交付物。标题:
chore(tests): selector recovery after <refactor description>
正文(从
candidates.json
生成,过滤
applied
为true的项):
markdown
undefined

Trigger

触发原因

<Link to the refactor PR / describe the redesign>
<重构PR链接/描述重新设计内容>

Summary

总结

  • N test files updated - M selectors changed
  • K tests deleted (feature removed) - L tests flagged for manual review
  • 更新N个测试文件 - 变更M个选择器
  • 删除K个测试(功能已移除) - L个测试标记为需人工评审

Per-file changes

按文件变更

<For each file: a table of line, old, new, score, screenshot URL>
<针对每个文件:行号、旧定位器、新定位器、评分、截图URL的表格>

Flagged for review

需评审的测试

<Tests where no candidate scored >= 3, with the inferred intent>
<无评分≥3候选选择器的测试,附带推断意图>

How to review

评审指南

  • Check each screenshot: does
    new
    point at the element
    old
    pointed at?
  • For score-3 candidates, verify the region scope is meaningful in the new design.
  • For flagged tests, decide: rewrite, delete, or accept a manual selector update.

Attach screenshots inline via your team's CI artifact URL pattern. See `references/recovery-scripts.md` for `apply-recovery.ts` (line-anchored) and `build-pr-body.ts`.

**Gate:** PR is reviewable in one sitting. Too large → split by area (one PR per page / component / test directory).

---
  • 查看每张截图:新定位器是否指向旧定位器对应的元素?
  • 对于评分3的候选选择器,验证区域范围在新设计中是否合理。
  • 对于标记的测试,决定:重写、删除或接受人工选择器更新。

通过团队的CI artifact URL模式将截图嵌入正文。参见`references/recovery-scripts.md`中的`apply-recovery.ts`(按行定位)和`build-pr-body.ts`。

**检查点:** PR可在一次评审中完成。若过大→按区域拆分(每个页面/组件/测试目录一个PR)。

---

Anti-Patterns

反模式

  1. Auto-applying score-0, -1, or -2 candidates. A score-2 is "we found some element." That is gambling, not recovery — and it is the usual cause of a suite-wide stability score dropping after a recovery. Auto-apply only score ≥ 3.
  2. Calling an ambiguous multi-match "score 3." If
    getByRole(...)
    returns >1 element it is not a 3 until region scoping narrows it to exactly one. Scoring it 3 and auto-applying ships a locator that resolves to the wrong element.
  3. Skipping the screenshots. A score-4 candidate can still point at the wrong element when the page has two regions with the same role + name. The per-change screenshot is the only check that catches semantic drift; confidence scores alone do not.
  4. Content-wide string replace instead of line-anchored edits.
    content.replace(oldLocator, …)
    hits the first occurrence only, collides on duplicate locators, and silently no-ops when the reporter's rendered string differs from the source expression. Edit the specific
    (file, line)
    .
  5. Trusting auto-extracted line numbers for POM-wrapped locators. The JSON reporter's
    error.location
    points at the failing line, which for a Page Object is the helper, not the test. Re-read the locator from the trace action or grep the POM source before applying.
  6. Recovering tests for deleted features. The refactor may have removed flows. Map deleted routes in Phase 2 and prune those tests — do not regenerate selectors for elements that no longer exist.
  7. Auto-merging the recovery PR in CI. The PR is the artifact; the whole point is a human eyeballing the per-change evidence. Auto-merge once tests pass defeats the purpose — a green suite with a selector pointing at the wrong-but-present element passes and erodes trust. Require a reviewer.
  8. Treating the PR as urgent. A failed suite feels urgent; a correctly recovered one is what matters. Time pressure produces score-2 replacements that quietly degrade the suite.

  1. 自动应用评分0、1或2的候选选择器。 评分2意味着“我们找到了某个元素”,这是碰运气而非修复——也是修复后套件整体稳定性评分下降的常见原因。仅自动应用评分≥3的候选选择器。
  2. 将歧义多匹配标记为“评分3”。 如果
    getByRole(...)
    返回>1个元素,需通过区域范围限定为单一匹配后才可评为3分。将其评为3分并自动应用会导致定位器解析到错误元素。
  3. 跳过截图。 当页面存在两个相同角色+名称的区域时,评分4的候选选择器仍可能指向错误元素。每项变更的截图是捕获语义漂移的唯一检查方式;仅靠置信度评分无法做到。
  4. 全局字符串替换而非按行编辑。
    content.replace(oldLocator, …)
    只会命中第一个匹配项,在重复定位器上发生冲突,且当报告中的渲染字符串与源代码表达式不同时会静默无操作。请编辑指定的
    (file, line)
  5. 信任POM封装定位器的自动提取行号。 JSON报告的
    error.location
    指向失败行,对于Page Object来说是助手代码而非测试代码。应用前需从跟踪操作中读取定位器,或在POM源代码中搜索渲染后的字符串。
  6. 为已删除功能的测试修复选择器。 重构可能已移除某些流程。在阶段2映射已删除的路由并删除对应测试——不要为已不存在的元素重新生成选择器。
  7. 在CI中自动合并修复PR。 PR本身就是交付物;核心目的是人工查看每项变更的证据。测试通过后自动合并会违背初衷——选择器指向错误但存在的元素时,套件会通过测试并逐渐降低信任度。需要求评审人员确认。
  8. 将PR视为紧急任务。 失效套件会让人感觉紧急,但正确修复才是关键。时间压力会导致应用评分2的替换项,悄悄降低套件质量。

Failure Modes

故障模式

SymptomLikely causeFix or check
identify-drift.ts
finds 0 failures despite red CI
Suite errored before producing the JSON report, or you parsed the wrong file
jq '.stats' .drift-recovery/results.json
; confirm
--reporter=json
redirected to the file
Extracted line points at a POM file, not the testLocator is wrapped in a Page ObjectRead the locator from the trace action, or grep the POM source for the rendered string
generate-candidates.ts
reads
undefined
for intent/route
Phase 3 output missing the inferred-intent / page-route fieldsPopulate them in Phase 3 — they are not in the reporter; the generator cannot infer them
New-DOM snapshot is missing client-rendered elementsSnapshotted before hydrationAdd
await page.waitForLoadState('networkidle')
before
ariaSnapshot()
Average stability score dropped after recoveryScore-2/CSS candidates auto-appliedRevert candidates with score < 3 in
candidates.json
; only role/label/testid should land (see scorer below)

症状可能原因修复或检查方法
identify-drift.ts
未找到任何失败,但CI显示失败
套件在生成JSON报告前出错,或解析了错误文件执行
jq '.stats' .drift-recovery/results.json
;确认
--reporter=json
已重定向到正确文件
提取的行指向POM文件而非测试文件定位器封装在Page Object中从跟踪操作中读取定位器,或在POM源代码中搜索渲染后的字符串
generate-candidates.ts
读取到
undefined
的意图/路由
阶段3输出缺失推断意图/页面路由字段在阶段3填充这些字段——报告中不包含这些信息,生成器无法自动推断
新DOM快照缺失客户端渲染元素快照在hydration前生成
ariaSnapshot()
前添加
await page.waitForLoadState('networkidle')
修复后平均稳定性评分下降自动应用了评分2/CSS候选选择器还原
candidates.json
中评分<3的候选选择器;仅应保留角色/标签/testid选择器(参见下方评分器)

Verification

验证

Run these on the recovery branch before opening the PR, smallest first:
bash
undefined
在打开PR前,在修复分支上按以下顺序执行验证:
bash
undefined

1. No applied candidate is below the stability floor (machine-checkable proxy for "ratcheted up")

1. 所有应用的候选选择器均不低于稳定性阈值(可机器检查的“质量提升”指标)

node references/score-candidates.mjs .drift-recovery/candidates.json
node references/score-candidates.mjs .drift-recovery/candidates.json

prints average score + count of applied rows with score < 3 — that count MUST be 0

打印平均评分+评分<3的已应用项数量——该数量必须为0

2. The recovered suite is green against the new build

2. 修复后的套件在新版本上运行通过

PLAYWRIGHT_TEST_BASE_URL=$PREVIEW_URL npx playwright test --reporter=json
| jq '.stats.unexpected' # must be 0 (flagged tests excluded via grep/skip)
PLAYWRIGHT_TEST_BASE_URL=$PREVIEW_URL npx playwright test --reporter=json
| jq '.stats.unexpected' # 必须为0(标记的测试已通过grep/skip排除)

3. The PR exists with the evidence body

3. PR已创建并包含证据正文

gh pr view --json title,body -q '.title' # contains "selector recovery"

`references/score-candidates.mjs` reads `candidates.json`, prints the average applied score and the count of `applied && score < 3` rows; a non-zero count means a low-confidence selector leaked in. Step 1 passing + step 2 returning `0` is the proof the recovery worked.

---
gh pr view --json title,body -q '.title' # 包含“selector recovery”

`references/score-candidates.mjs`读取`candidates.json`,打印已应用项的平均评分及`applied && score < 3`的项数;非零数值意味着低置信度选择器被混入。步骤1通过+步骤2返回`0`是修复成功的证明。

---

Done When

完成标准

  • Every in-scope test passes on the new build, is flagged for human review with a clear reason, or is deleted because its feature is gone.
  • references/score-candidates.mjs candidates.json
    reports 0 applied candidates with score < 3.
  • The PR is open (
    gh pr view
    succeeds) with per-change evidence: confidence score and screenshot per change, grouped by file.
  • npx playwright test --reporter=json | jq '.stats.unexpected'
    returns
    0
    on the recovery branch.
  • A short note is added to
    .agents/qa-project-context.md
    describing the refactor and any new test patterns introduced.
  • 所有范围内的测试在新版本上通过、被标记为需人工评审(附带明确原因),或因功能已移除被删除。
  • references/score-candidates.mjs candidates.json
    报告0个评分<3的已应用候选选择器。
  • PR已创建(
    gh pr view
    执行成功),并附带每项变更的证据:置信度评分和截图,按文件分组。
  • 在修复分支上执行
    npx playwright test --reporter=json | jq '.stats.unexpected'
    返回
    0
  • .agents/qa-project-context.md
    中添加简短说明,描述重构内容及引入的新测试模式。

Reference Files (in
references/
)

参考文件(位于
references/

  • recovery-scripts.md — the full playbook with corrected, runnable scripts: aria-snapshot capture,
    identify-drift.ts
    (populates intent + route),
    generate-candidates.ts
    (region-scoping ladder, true score-3), line-anchored
    apply-recovery.ts
    , and
    build-pr-body.ts
    . Includes the Cypress-adaptation note.
  • score-candidates.mjs — tiny stability scorer; prints the average applied score and the count of applied rows below score 3. Used by Verification and Done When.
  • recovery-scripts.md — 完整操作指南及可运行的修正脚本:aria快照捕获、
    identify-drift.ts
    (填充意图+路由)、
    generate-candidates.ts
    (区域范围优先级、真实评分3)、按行定位的
    apply-recovery.ts
    build-pr-body.ts
    。包含Cypress适配说明。
  • score-candidates.mjs — 小型稳定性评分器;打印已应用项的平均评分及评分<3的已应用项数。用于验证及完成标准检查。

Related Skills

相关技能

  • test-reliability — runtime per-test healing. Use for one flaky test, not a refactor-driven mass update. Shares the 0–5 stability rubric.
  • playwright-automation — writing new tests from scratch. Use when the refactor removed enough features that tests should be rewritten, not patched.
  • test-migration — switching frameworks (Selenium → Playwright). A migration re-records tests; it is not selector drift, even though both touch many tests at once.
  • visual-testing — had this run on every PR, the refactor's visual diff would have flagged before merge. Recovery is the fallback when that coverage is missing.
  • ci-cd-integration — wires the recovery PR's validation step into CI.
  • test-reliability — 运行时单测试修复。适用于单个不稳定测试,而非重构导致的批量更新。共享0-5稳定性评分标准。
  • playwright-automation — 从头编写新测试。当重构移除大量功能,测试需重写而非修补时使用。
  • test-migration — 框架切换(Selenium → Playwright)。迁移是重新录制测试;不属于选择器漂移,尽管二者都会同时修改大量测试。
  • visual-testing — 如果在每个PR上运行该技能,重构的视觉差异会在合并前被标记。当该覆盖缺失时,修复是备选方案。
  • ci-cd-integration — 将修复PR的验证步骤接入CI。