test-audit

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Test Audit

测试审计

Three modes, one value bar. Authoring mode gates every new or changed test at write time. Audit mode runs focused sweeps for tests that re-assert source, duplicate stronger proof, couple behavior to implementation, or keep test-only production seams alive. Campaign mode reviews every test file with a per-test verdict when the maintainer asks to prune the whole suite; see Campaign mode. Optimize for confidence, not deletion count. Land one coherent batch per PR; continue broad audits as separate follow-ups.

The repository's own rules live in the "Testing" section of

AGENTS.md

. This skill is the procedure for applying them; when the two disagree,

AGENTS.md

wins and this file needs an update.

三种模式，一套价值标准。**编写模式（Authoring mode）**在编写时对每个新增或修改的测试执行门禁。**审计模式（Audit mode）**聚焦排查以下问题：重复断言源码、重复提供更强的证明、将行为与实现耦合，或让仅供测试使用的 production seam 继续存在。**全量清理模式（Campaign mode）**在维护者要求精简整个测试套件时，审查每个测试文件并针对每个测试给出结论；参见全量清理模式。目标是增强信心，而不是追求删除数量。每个 PR 落地一批连贯的改动；范围广泛的审计应作为后续独立工作继续进行。

本仓库自身的规则位于

AGENTS.md

的

Testing

部分。此 skill 是应用这些规则的操作流程；两者发生冲突时，以

AGENTS.md

为准，并需要更新本文件。

Authoring gate

编写门禁

Before adding a test, answer four questions. A missing answer means do not add it yet:

What observable behavior, invariant, or independent contract does it protect?
What credible regression makes it fail?
Why does existing coverage not already catch that failure? Each contract has one primary owner at the strongest boundary (usually the service or the MCP tool handler, not the repository or a private helper). Another layer needs its own distinct risk. Prefer extending a table-driven case or a shared factory (
```
ga4-test-fixtures.ts
```
,
```
tool-test-support.ts
```
) over a near-duplicate test, and consolidate duplicated setup in the same change.
Does it need a production seam (export, flag, wrapper, injection hook) that no production caller needs? If yes, test at the real boundary instead. Grep the whole repo for callers, not just
```
src/
```
;
```
scripts/
```
counts.

Then check the test against every junk pattern; a match fails the gate unless the retention bar names the contract it independently guards. A test that would break under behavior-preserving refactoring is asserting implementation, not behavior; rewrite it at the owning boundary before landing it.

Bug regression tests must fail on the pre-fix code for the intended reason and pass after the fix. A regression test that never demonstrably failed proves the mock, not the fix. One regression at the owner boundary covers the bug; do not replay it at every layer it crosses.

在添加测试前，回答四个问题。只要有一个问题没有答案，就暂时不要添加该测试：

它保护的是哪种可观察行为、不变量或独立契约？
哪种可信的回归会导致它失败？
为什么现有覆盖无法捕获该失败？每项契约都应有一个位于最强边界的 primary owner（通常是 service 或 MCP tool handler，而不是 repository 或 private helper）。其他层需要具备自身的独立风险。应优先扩展 table-driven case 或 shared factory（
```
ga4-test-fixtures.ts
```
、
```
tool-test-support.ts
```
），而不是新增近似重复的测试，并在同一次改动中整合重复的 setup。
它是否需要某个生产调用方并不需要的 production seam（
```
export
```
、flag、wrapper、注入 hook）？如果是，应改为在真实边界处进行测试。搜索整个仓库中的调用方，不要只查
```
src/
```
；
```
scripts/
```
也算。

然后对照每一个低价值模式检查该测试；除非保留标准明确指出它所独立守护的契约，否则命中任一模式都视为未通过门禁。如果一个测试会在保持行为不变的 refactor 下发生断裂，它断言的是实现，而不是行为；应先在 owner 边界重写该测试，然后才能合入。

缺陷回归测试必须因预期原因在修复前代码上失败，并在修复后通过。一个从未被明确证明会失败的回归测试，证明的是 mock，而不是修复本身。一项缺陷应由 owner 边界上的回归测试覆盖；不要在其穿过的每一层重复该测试。

Junk patterns

低价值模式

The shared checklist for both modes. The authoring gate rejects a new test that matches one; audits hunt for existing tests that do.

Repo-specific (each is a

AGENTS.md

rule):

per-test
```
await import()
```
or
```
vi.resetModules()
```
without a comment explaining which module-level state must reset;
a production class re-declared inside a test instead of imported from a leaf module (
```
ga4Errors.ts
```
,
```
gscErrors.ts
```
are the pattern);
```
mockReset
```
/
```
mockClear
```
ceremonies in
```
beforeEach
```
(Vitest
```
clearMocks
```
is already on);
fixtures longer than the assertions that read them, or inline copies of a shape that already has a factory;
Zod schemas or third-party libraries re-tested through the module under test;
an output-schema round-trip repeated in every happy path;
Drizzle builder chains mocked (
```
select().from().where()
```
with
```
vi.fn()
```
chains); repositories are tested through services or real SQL;
argument forwarding to a mock asserted when the mapping is not the contract (billing params and telemetry events are the contract; internal helper arguments are not).

General:

assertion-free coverage probes;
self-comparisons, and expected values produced by the helper or renderer under test;
copied fixtures, inventories, manifests, or export lists;
exact source, import, or string greps;
private predicate or call-shape tests duplicated at real boundaries;
duplicate invocations of the same contract;
tests whose only purpose is preserving test-only exports or wrappers;
dead production code whose only callers are tests;
mocks that implement the asserted behavior, or one identical mock standing in for different APIs;
negative controls that pass for an unrelated reason, such as a denial from a different guard or a rejection the production path never reaches;
names or fixtures that promise more than the input exercises.

以下清单同时适用于两种模式。编写门禁会拒绝匹配任一模式的新测试；审计则查找存在这些问题的现有测试。

仓库特定规则（每一条都对应一项

AGENTS.md

规则）：

在单个测试中使用
```
await import()
```
或
```
vi.resetModules()
```
，却没有注释说明必须重置哪项模块级状态；
在测试内重新声明生产类，而不是从 leaf module 导入（
```
ga4Errors.ts
```
、
```
gscErrors.ts
```
是此模式的范例）；
在
```
beforeEach
```
中进行
```
mockReset
```
/
```
mockClear
```
仪式性清理（Vitest 的
```
clearMocks
```
已启用）；
fixture 比读取它的断言还长，或内联复制了已有 factory 的数据结构；
通过被测模块重复测试 Zod schema 或第三方库；
在每条 happy path 中重复执行 output-schema 往返验证；
mock Drizzle builder chain（使用
```
vi.fn()
```
链模拟
```
select().from().where()
```
）；repository 应通过 service 或真实 SQL 测试；
当映射关系并非契约时，却断言向 mock 转发参数（billing params 和 telemetry events 才是契约；internal helper 的参数不是）。

通用模式：

无断言的 coverage probe；
自比较，以及预期值由被测 helper 或 renderer 生成；
复制的 fixture、inventory、manifest 或 export 列表；
对源码、import 或字符串进行精确 grep；
在真实边界重复测试 private predicate 或 call-shape；
重复调用同一契约；
唯一目的只是保留 test-only export 或 wrapper 的测试；
唯一调用方只有测试的 dead production code；
实现了被断言行为的 mock，或用同一个完全相同的 mock 代替不同 API；
因无关原因而通过的 negative control，例如由另一个 guard 拒绝，或触达生产路径永远不会发生的 rejection；
名称或 fixture 所承诺的内容超过输入实际覆盖的范围。

Value bar

价值标准

Tests justify their maintenance cost by protecting behavior, a credible regression, or an independently meaningful contract. In an audit, an existing test that must change for behavior-preserving source reorganization is suspect, not automatically deletable; the authoring gate still rejects new ones.

Before judging a candidate, read the complete test and its production owner, the entry point, callers, callees, sibling implementations, overlapping tests, and relevant git history. When the test claims dependency-backed behavior (DataForSEO response shapes, Autumn, Better Auth, Drizzle), inspect the dependency source or types directly.

测试应通过保护行为、提供可信回归或守护具有独立意义的契约，来证明其维护成本的合理性。在审计中，如果一个现有测试必须因保持行为不变的源码重组而修改，就应将其视为可疑对象，而不是自动删除；编写门禁仍会拒绝新增此类测试。

在判断候选项之前，完整阅读该测试及其 production owner、入口点、调用方、被调用方、同级实现、重叠测试，以及相关 git 历史。当测试声称覆盖 dependency-backed behavior（DataForSEO 响应结构、Autumn、Better Auth、Drizzle）时，直接检查 dependency 的源码或类型。

Discovery

发现

Keep discovery read-only and report evidence before editing. For broad scope, run parallel read-only lanes (Explore subagents work well):

```
src/server/mcp/
```
and
```
src/server/mcp/tools/
```
;

src/server/lib/

src/server/auth/

src/server/billing/

src/server/workflows/

src/serverFunctions/

;

```
src/server/features/
```
;
```
src/client/
```
,
```
src/shared/
```
,
```
src/lib/
```
,
```
src/types/
```
,
```
scripts/
```
,
```
web/tests/
```
, plus a cross-cutting grep sweep for the repo-specific patterns above.

Prefer a few high-confidence candidates over a large speculative inventory.

发现阶段应保持只读，并在编辑前报告证据。对于广泛范围，可并行运行只读通道（Explore subagents 效果很好）：

```
src/server/mcp/
```
和
```
src/server/mcp/tools/
```
；

src/server/lib/

、

src/server/auth/

、

src/server/billing/

、

src/server/workflows/

、

src/serverFunctions/

；

```
src/server/features/
```
；
```
src/client/
```
、
```
src/shared/
```
、
```
src/lib/
```
、
```
src/types/
```
、
```
scripts/
```
、
```
web/tests/
```
，以及对上述仓库特定模式执行一次跨目录 grep。

优先选择少量高置信度候选项，而不是大量推测性 inventory。

Campaign mode

全量清理模式

Use only when the maintainer explicitly asks for a suite-wide prune. Run the pipeline in two phases with disjoint file slices so agents never edit the same file:

Review, read-only. Split every
```
*.test.ts
```
into 6-8 slices balanced by line count and grouped by directory. One reviewer per slice reads each test file and its production owner completely and writes a report with a table per file: test name, DELETE / MERGE / KEEP, one-line evidence naming the code change that would fail it or the test that already owns it. The report also lists test-only production seams (with callers checked in
```
src/
```
,
```
scripts/
```
,
```
tests/badseo/
```
,
```
web/
```
), test support that becomes unused, whole files to delete, and a "risky calls" section. Reviewers do not edit.
Apply, per slice. One applier per slice re-verifies each DELETE and MERGE against the source before acting, keeps anything whose evidence does not hold, folds MERGE rows into table-driven siblings, deletes newly unused fixtures and mocks, and skips every risky item, listing it back for the maintainer. If a deletion's justification cites a test in another slice, the applier confirms that test still exists at the end of its run.

Rules that hold throughout a campaign:

The only permitted non-test edit is removing an
```
export
```
keyword from a helper whose remaining callers are all tests. Knip fails
```
ci:check
```
on the orphaned export otherwise. Never remove parameters, delete branches, or change behavior in the same change, even when a test looks like the only reason the code exists; report those as follow-ups.
Route modules under
```
src/routes/
```
and barrel files keep their exports.
Use
```
pnpm exec prettier --write <files>
```
;
```
pnpm format:write
```
ignores arguments and formats the whole repository.
After all slices land, run the full suite in normal order and with two or three
```
--sequence.shuffle.tests --sequence.seed=<n>
```
runs. Removing reset ceremony can surface latent order dependence; fix it by setting the mock's default in
```
beforeEach
```
, never by restoring the ceremony.

仅当维护者明确要求对整个测试套件进行精简时，才使用此模式。流程分为两个阶段，使用互不重叠的文件分片，确保 Agent 永远不会编辑同一文件：

审查，只读。 将每个
```
*.test.ts
```
按行数均衡划分为 6–8 个分片，并按目录分组。每个分片由一名 reviewer 负责；reviewer 需完整阅读该分片中的每个测试文件及其 production owner，然后为每个文件编写报告，其中包含一张表：测试名称、DELETE / MERGE / KEEP，以及一行证据，说明什么代码变更会导致该测试失败，或哪个测试已经拥有该契约。报告还应列出 test-only production seam（需检查
```
src/
```
、
```
scripts/
```
、
```
tests/badseo/
```
、
```
web/
```
中的调用方）、将变为未使用的 test support、可整文件删除的测试文件，以及“高风险项”部分。Reviewer 不执行编辑。
应用，按分片执行。 每个分片由一名 applier 负责。applier 在采取行动前，需根据源码重新验证每个 DELETE 和 MERGE；保留证据不成立的项，将 MERGE 项合并到 table-driven sibling 测试中，删除新近未使用的 fixture 和 mock，并跳过所有高风险项，将其列入清单交回维护者。如果某项删除理由引用了另一个分片中的测试，applier 需在本次运行结束时确认该测试仍然存在。

campaign 全程适用的规则：

唯一允许的非测试改动，是从剩余调用方全部为测试的 helper 中移除
```
export
```
关键字。否则，Knip 会因孤立的 export 而导致
```
ci:check
```
失败。即使某个测试看起来是某段代码存在的唯一原因，也绝不要在同一次改动中删除参数、删除分支或改变行为；应将这些情况报告为后续事项。
```
src/routes/
```
下的路由模块和 barrel 文件必须保留其 exports。

使用

pnpm exec prettier --write <files>

；

pnpm format:write

会忽略参数并格式化整个仓库。

所有分片落地后，按正常顺序运行完整测试套件，并额外运行两到三次
```
--sequence.shuffle.tests --sequence.seed=<n>
```
。移除重置操作可能暴露潜在的顺序依赖；应通过在
```
beforeEach
```
中设置 mock 的默认值来修复，绝不能通过恢复这些重置操作来解决。

Retention bar

保留标准

Keep a test when it independently enforces a public API, MCP tool contract, Zod boundary schema, config, migration, storage (both SQLite and Postgres), billing, auth, security, default, prompt-byte, or package contract. Also keep:

call ordering when order is observable behavior;
regressions with a credible failure mode;
source inspection when it is the cheapest independent guard: it fails when the contract changes (a user-facing key, byte, or path) and survives an identifier-only refactor. The pinned SAM skill roster in
```
samSkills.test.ts
```
and the unique-index parity check in
```
src/db/schema-parity.test.ts
```
are this kind;
a retained test that fails on the baseline: treat it as a possible product bug, reproduce it, and repair the owner rather than deleting it.

Static or slow is not a deletion reason. A test that resembles implementation may still be the independent contract; prove otherwise before removing it.

当测试能够独立约束以下内容时，应予以保留：public API、MCP tool 契约、Zod 边界 schema、config、migration、storage（SQLite 和 Postgres 两者）、billing、auth、security、default、prompt-byte 或 package 契约。还应保留：

顺序可观察时，对调用顺序的断言；
具有可信失败模式的 regression；
当源码检查是最经济的独立 guard 时：当契约（user-facing key、byte 或 path）发生变化时它会失败，并且在仅重构 identifier 时仍然有效。
```
samSkills.test.ts
```
中锁定的 SAM skill 清单，以及
```
src/db/schema-parity.test.ts
```
中的唯一索引一致性检查，均属于这一类；
保留的测试在 baseline 上失败时：将其视为潜在的产品 bug，复现问题并修复 owner，而不是删除测试。

静态或运行缓慢都不是删除理由。一个看似与实现相似的测试，仍可能是独立契约；删除前必须先证明它确实不是独立契约。

Candidate evidence

候选项证据

Record every field before editing. A missing field means the candidate is not ready for deletion:

exact test name and location;
what failure it can actually detect;
non-test callers of the covered production or support seam, including
```
scripts/
```
,
```
tests/badseo/
```
, and
```
web/
```
(a profiling script is still a caller);
stronger remaining owner-boundary proof, or why no proof is needed;
relevant history and the reason the test or seam exists;
production or test-support deletion unlocked;
risk and the focused validation command.

编辑前记录以下每个字段。只要缺少一个字段，该候选项就尚未达到可删除状态：

测试的准确名称和位置；
它实际能够检测到的故障；
所覆盖 production 或 support seam 的非测试调用方，包括
```
scripts/
```
、
```
tests/badseo/
```
和
```
web/
```
（profiling script 也算调用方）；
更强有力的剩余 owner-boundary proof，或说明为何不需要 proof；
相关历史，以及该测试或 seam 存在的原因；
可随之删除的生产代码或 test support；
风险以及聚焦验证命令。

Edit shape

编辑方式

Choose one coherent owner-boundary batch. Delete obsolete test-only exports, wrappers, and dead production paths instead of preserving aliases (Knip will flag survivors in

pnpm ci:check

). Move retained regressions to their canonical owners. Consolidate repeated assertions into one table-driven case.

Prefer net-negative production LOC. Do not add replacement tests that restate the same implementation, and do not convert uncertain candidates into cleanup to increase deletion counts.

选择一个连贯的 owner-boundary 批次。删除过时的 test-only export、wrapper 和 dead production path，不要保留 alias（Knip 会在

pnpm ci:check

中标记残留项）。将保留的 regression 迁移到其 canonical owner。将重复断言整合到一个 table-driven case 中。

优先实现 production LOC 净减少。不要添加只是重述相同实现的替代测试，也不要为了提高删除数量而把不确定候选项强行转化为清理对象。

Validation

验证

Never edit tests while Vitest is running in the checkout.

Run the owner and sibling tests:
```
pnpm exec vitest run <path-or-filter>
```
.
For a removed source grep or static assertion, run the executable that owns the real contract (for example
```
pnpm sync-plugin-skills
```
for plugin skill drift).
```
pnpm format:write
```
, then
```
pnpm ci:check
```
(prettier, knip, tsc, oxlint, plugin-skill sync). Knip failing on a now-unused export means delete the export, not re-add a test.
Run the full suite once:
```
pnpm test
```
.
Inspect
```
git diff --numstat
```
; report production/tooling separately from tests and test support.

Vitest 在当前 checkout 中运行时，绝不要编辑测试。

运行 owner 和 sibling 测试：
```
pnpm exec vitest run <path-or-filter>
```
。
对于已移除的源码 grep 或静态断言，运行真正拥有该契约的可执行程序（例如，针对 plugin skill 漂移运行
```
pnpm sync-plugin-skills
```
）。
运行
```
pnpm format:write
```
，然后运行
```
pnpm ci:check
```
（prettier、knip、tsc、oxlint、plugin-skill sync）。如果 Knip 因某个现已无用的 export 而失败，应删除该 export，而不是重新添加测试。
完整测试套件运行一次：
```
pnpm test
```
。
检查
```
git diff --numstat
```
；将生产代码或工具链与测试及 test support 分开报告。

Landing

合入

Commit, push, or open a PR only when authorized. Use the

merge-ready

skill for the review and PR flow. If a verified finding exposes a recurring invariant that

.greptile/

does not capture, use

maintain-greptile-rules

; do not promote one-off cleanups into permanent rules. Log repository friction met along the way with

papercuts

仅在获得授权后 commit、push 或打开 PR。审查和 PR 流程使用

merge-ready

skill。如果某个已验证的发现暴露出

.greptile/

未能捕获的反复出现的不变量，使用

maintain-greptile-rules

；不要将一次性清理提升为永久规则。沿途使用

papercuts

记录遇到的仓库摩擦问题。

Handoff

交接

Report:

removed low-value categories and the root cause behind them;
production owner simplifications;
retained false positives and why they remain valuable;
focused and full proof actually run, with real output;
production versus test LOC;
PR and merge state;
named follow-ups.

报告：

已删除的低价值测试类别及其根本原因；
production owner 的简化情况；
保留的误报及其仍有价值的原因；
实际运行的聚焦验证和完整验证，以及实际输出；
production 代码与测试代码的 LOC 变化；
PR 和 merge 状态；
明确列出的后续事项。