Back to Details

test-doctor

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Test Doctor

Test Doctor

Treat tests as product protection, not implementation proof. Ask of each test: "If this fails, which user-visible capability or documented contract is probably broken?" If there is no concrete answer, require evidence that the test deserves to remain.
将测试视为产品保护，而不是实现正确性的证明。问每个测试：“如果此测试失败，可能破坏了哪个用户可见能力或文档化契约？”如果没有具体答案，则要求提供证据，证明该测试值得保留。

Select the operating mode

选择工作模式

  • For audit, analysis, diagnosis, or planning requests, inspect and report without editing files.
  • For explicit change requests, audit first, then make the smallest test changes that preserve or improve regression protection.
  • Do not change production behavior merely to satisfy a test unless the user explicitly requests that fix.
  • Read repository instructions and the relevant design or contract documents before evaluating intended behavior. Report a contract mismatch instead of guessing which side is correct.
  • 对于审计、分析、诊断或规划请求，仅检查并报告，不编辑文件。
  • 对于明确的变更请求，先进行审计，然后进行能够保留或增强回归保护的最小测试变更。
  • 除非用户明确要求修复，否则不要仅为满足测试而更改生产行为。
  • 在评估预期行为之前，阅读仓库说明以及相关的设计或契约文档。如果发现契约不匹配，应报告该问题，而不是猜测哪一方正确。

Build an evidence base

建立证据基础

  1. Locate test files, fixtures, helpers, configuration, package scripts, and CI workflows.
  2. Identify source-package boundaries and public entry points.
  3. Read relevant feature, architecture, API, and bug-fix documentation.
  4. Inspect focused git history when it explains why a suspicious test exists.
  5. Record the available test commands and their scope. Run only commands needed to verify a finding or requested change.
For a large repository, start with package-level inventory and high-risk paths. Expand to individual tests only where evidence indicates low signal, duplication, fragility, or missing protection.
  1. 查找测试文件、fixtures、helpers、配置、package scripts 和 CI workflows。
  2. 识别源码包边界和公共入口点。
  3. 阅读相关的功能、架构、API 和 bug-fix 文档。
  4. 当聚焦的 git history 能够解释可疑测试存在的原因时，检查相关历史。
  5. 记录可用的测试命令及其覆盖范围。仅运行验证某项发现或完成所请求变更所需的命令。
对于大型仓库，先从包级清单和高风险路径入手。仅在证据表明存在低信号、重复、脆弱或保护缺失时，才扩展到单个测试。

Map tests to protected capabilities

将测试映射到受保护的能力

Create a compact map with one row per test file or coherent test group:
FieldMeaning
TestFile and test or suite name
TypeUnit, integration, contract, component, or E2E
Protected capabilityUser behavior or documented contract detected by failure
BoundaryReal dependencies and substituted dependencies
RiskImpact and likelihood of regression
CostRuntime, setup complexity, brittleness, and maintenance burden
ConfidenceHigh, medium, or low, with evidence
Mark the protected capability as 
unclear
when evidence is insufficient. Do not invent intent from test names alone.
创建一个紧凑的映射表，每个测试文件或内聚的测试组占一行：
字段含义
测试文件及测试或测试套件名称
类型单元、集成、契约、组件或 E2E
受保护的能力通过失败可检测到的用户行为或文档化契约
边界真实依赖和被替换的依赖
风险回归的影响和发生可能性
成本运行时间、设置复杂度、脆弱性和维护负担
置信度高、中或低，并附有证据
当证据不足时，将受保护的能力标记为 
unclear
。不要仅凭测试名称推测其意图。

Evaluate test value

评估测试价值

Use this as a relative heuristic, not a fabricated numeric measurement:
text
Test value = user impact × regression risk × bug-detection strength
             --------------------------------------------------------
                            maintenance cost
Rate each factor as high, medium, or low and explain disputed ratings.
Prioritize tests that protect:
  • User workflows, data correctness, security, compatibility, and recovery.
  • Public APIs, provider interfaces, persistence formats, migrations, and cross-package contracts.
  • Core runtime orchestration, file handling, session state, and important failure paths.
  • Boundary conditions that have failed before or are easy to regress.
Discount tests that primarily protect:
  • Private helpers, internal object shape, exact call order, or incidental function counts.
  • Mock configuration rather than observable behavior.
  • Static formatting with no contract or user impact.
  • Duplicate paths already protected more directly at a stronger boundary.
Increase the maintenance-cost rating for heavy snapshots, excessive mocks, fragile selectors, internal API coupling, large duplicated fixtures, timing dependence, and slow environment setup.
将此公式作为相对评估启发式，而不是虚构的数值指标：
text
测试价值 = 用户影响 × 回归风险 × 缺陷检测能力
             --------------------------------------------------------
                            维护成本
将每个因素评为高、中或低，并解释存在争议的评级。
优先保护以下方面的测试：
  • 用户工作流、数据正确性、安全性、兼容性和恢复能力。
  • 公共 API、provider interfaces、持久化格式、迁移以及跨包契约。
  • 核心运行时编排、文件处理、会话状态和重要的失败路径。
  • 曾经失败过或容易发生回归的边界条件。
降低以下测试的优先级：
  • 仅保护私有 helpers、内部对象结构、精确调用顺序或附带函数数量的测试。
  • 验证 Mock 配置而非可观察行为的测试。
  • 不涉及契约或用户影响的静态格式检查。
  • 已在更强边界上得到更直接保护的重复路径。
对于大量 snapshots、过度 Mock、脆弱 selectors、内部 API 耦合、大型重复 fixtures、依赖时序和缓慢环境设置，应提高维护成本评级。

Detect test smells

检测测试坏味道

Implementation coupling

实现耦合

Flag assertions about private methods, internal state, exact call order, or incidental calls. Recommend assertions on returned state, persisted data, emitted public events, rendered semantics, or externally observable errors.
Do not flag interaction assertions automatically. Keep them when the interaction itself is the contract, such as authorization checks, idempotency, transaction boundaries, or required protocol calls.
标记针对私有方法、内部状态、精确调用顺序或附带调用的断言。建议改为断言返回状态、持久化数据、发出的公共事件、渲染语义或外部可观察错误。
不要自动将 interaction assertions 视为问题。当 interaction 本身就是契约时应予保留，例如 authorization checks、idempotency、transaction boundaries 或必需的 protocol calls。

Excessive mocking

过度 Mock

Flag suites where most collaborators are mocked and the assertions only restate mock behavior. Prefer the narrowest realistic boundary:
  • Real domain logic and stores.
  • In-memory databases or filesystems when faithful.
  • Small deterministic fakes for network or model providers.
  • Mocks only at expensive, nondeterministic, destructive, or externally owned boundaries.
标记那些大部分 collaborators 都被 Mock，且断言只是在复述 Mock 行为的测试套件。优先采用最窄且真实的边界：
  • 真实的 domain logic 和 stores。
  • 在能够忠实模拟时使用内存数据库或文件系统。
  • 对网络或 model providers 使用小型、确定性的 Fakes。
  • 仅在昂贵、不确定、破坏性或由外部拥有的边界使用 Mocks。

Dead or vacuous tests

失效或空洞测试

Find skipped or focused tests, empty tests, tautological assertions, obsolete APIs, unreachable setup, tests that cannot fail after the exercised code is removed, and snapshots that accept meaningless churn.
Classify skipped tests carefully: a linked known issue may be deferred coverage rather than dead code.
查找被跳过或仅聚焦的测试、空测试、同义反复断言、过时 API、不可达的设置、删除所执行代码后仍无法失败的测试，以及接受无意义变化的 snapshots。
谨慎分类被跳过的测试：关联到已知问题的测试可能是被延期的覆盖范围，而非失效代码。

Duplicate coverage

重复覆盖

Group tests by capability and failure mode, not by similar names. Call coverage duplicate only when removing one test would not reduce detection of a distinct regression. Prefer merging repeated setup and closely related lifecycle behavior into one coherent integration or contract suite.
按能力和失败模式对测试分组，而不是按相似名称分组。仅当移除某个测试不会减少对某项独立回归的检测能力时，才将其判定为重复覆盖。优先将重复设置和紧密相关的生命周期行为合并为一个内聚的集成或契约测试套件。

Snapshot overuse

过度使用 Snapshot

Keep snapshots for stable, reviewable contracts such as accessibility output, serialized protocols, or critical UI states. Replace broad generated markup and noisy component snapshots with focused semantic assertions.
对于稳定且易于审查的契约，例如 accessibility output、序列化 protocols 或关键 UI states，应保留 snapshots。将宽泛的生成式 markup 和噪声较多的组件 snapshots 替换为聚焦语义的断言。

Missing negative and boundary coverage

缺失负向和边界覆盖

Look for happy-path-only suites around validation, permissions, cancellation, retries, partial failure, concurrency, migration, recovery, and unknown variants. Recommend additions only when a concrete contract and credible regression are identifiable.
查找仅覆盖 happy path 的测试套件，尤其是涉及 validation、permissions、cancellation、retries、partial failure、concurrency、migration、recovery 和 unknown variants 的场景。仅当能够识别具体契约和可信回归风险时，才建议新增测试。

Choose a treatment

选择处理方式

Choose exactly one primary action per finding:
  • Keep: The test protects distinct behavior at reasonable cost.
  • Delete: The test is dead, vacuous, or fully redundant and retained coverage is identified.
  • Merge: Several tests protect one capability with duplicated setup or assertions.
  • Move: The behavior belongs at a different boundary, usually integration or contract level.
  • Rewrite: The capability matters, but the current assertions protect the wrong abstraction.
  • Promote: Repeated lower-level cases reveal a missing public contract or end-to-end workflow test.
  • Add: A high-risk capability has no effective regression protection.
Never recommend deletion without naming the retained protection or explicitly identifying the accepted coverage gap.
为每项发现选择且仅选择一个主要操作：
  • 保留：该测试以合理成本保护了独立行为。
  • 删除：该测试失效、空洞或完全冗余，并且已确定保留的覆盖范围。
  • 合并：多个测试通过重复设置或断言来保护同一项能力。
  • 迁移：该行为属于其他边界，通常是集成或契约层级。
  • 重写：该能力很重要，但当前断言保护了错误的抽象层级。
  • 提升：重复出现的底层用例表明缺少公共契约或端到端工作流测试。
  • 新增：高风险能力没有有效的回归保护。
推荐删除测试时，必须指出保留的保护范围，或明确说明已接受的覆盖缺口。

Handle agent and Electron applications

处理 Agent 和 Electron 应用

When relevant, inspect these high-value contracts:
  • Provider: initialization, model discovery, streaming, tool calls, reasoning support, usage reporting, cancellation, and errors.
  • Runtime: user input, context assembly, model request, tool execution, persistence, and final response.
  • Storage: session creation, message append, history loading, migration, recovery, and corruption handling.
  • Skills and plugins: install, discovery, loading, execution, permissions, and failure isolation.
  • Desktop UI: renderer behavior through public preload or IPC contracts, accessible interaction, persisted state, and platform differences.
Keep provider-specific cases only where providers genuinely differ. Put shared behavior in a reusable contract suite.
在相关场景中，检查以下高价值契约：
  • Provider：初始化、模型发现、streaming、tool calls、reasoning support、用量报告、取消和错误处理。
  • Runtime：用户输入、上下文组装、模型请求、工具执行、持久化和最终响应。
  • Storage：会话创建、消息追加、历史加载、迁移、恢复和损坏处理。
  • Skills 和 plugins：安装、发现、加载、执行、权限和故障隔离。
  • Desktop UI：通过公共 preload 或 IPC 契约体现的 renderer 行为、无障碍交互、持久化状态和平台差异。
仅在 providers 确实存在差异时保留 provider-specific 用例。将共享行为放入可复用的 contract suite。

Validate requested changes

验证所请求的变更

Before editing, identify the capability protected before and after the change. Preserve evidence of replacement coverage when deleting or merging tests.
Verify proportionally:
  1. Run the smallest affected test target.
  2. Run the owning package's broader target only when boundary changes justify it.
  3. Use existing coverage or mutation tooling when it answers a disputed finding.
  4. Introduce a targeted mutation only when authorized and safe; verify the test fails, restore the source exactly, then verify the test passes.
Do not claim stronger protection merely because tests pass. Explain which realistic regressions the assertions can detect.
编辑之前，识别变更前后所保护的能力。删除或合并测试时，保留替代覆盖范围的证据。
按相应比例进行验证：
  1. 运行受影响的最小测试目标。
  2. 仅当边界变更确有依据时，运行所属 package 的更大测试目标。
  3. 当现有 coverage 或 mutation 工具能够解决有争议的发现时，使用这些工具。
  4. 仅在获得授权且操作安全时引入定向 mutation；验证测试会失败，精确还原源码，然后验证测试能够通过。
不要仅因测试通过就声称保护能力得到增强。说明这些断言能够检测哪些现实中的回归。

Report results

报告结果

Lead with actionable findings ordered by capability risk. For each finding, include:
text
Priority:
Location:
Protected capability:
Evidence:
Problem:
Primary action:
Replacement or retained coverage:
Expected benefit:
Confidence:
Then provide:
  1. A summary of high-, medium-, and low-value coverage. Use counts only when the inventory is complete.
  2. A migration plan ordered so replacement coverage lands before deletion.
  3. Missing high-risk coverage backed by documented behavior.
  4. Verification performed and remaining evidence gaps.
If there are no actionable findings, say so directly and list residual coverage or verification gaps. Separate observed facts from recommendations and label uncertain inferences.
首先列出可执行的发现，并按能力风险排序。每项发现应包括：
text
优先级：
位置：
受保护的能力：
证据：
问题：
主要操作：
替代覆盖范围或保留的覆盖范围：
预期收益：
置信度：
随后提供：
  1. 高、中、低价值覆盖范围的摘要。仅在清单完整时才使用计数。
  2. 按替代覆盖范围先落地、测试后删除的顺序安排迁移计划。
  3. 有文档化行为作为依据的缺失高风险覆盖范围。
  4. 已执行的验证和剩余证据缺口。
如果没有可执行的发现，应直接说明，并列出残余覆盖缺口或验证缺口。区分观察到的事实与建议，并标注不确定的推断。