gjalla-test-audit

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Test Audit

测试审计

Deep review of test suite quality to find tests that give false confidence, encode bugs, duplicate coverage, or are so heavily mocked they can't catch real regressions.

深度审查测试套件质量，找出那些会带来虚假信心的测试，包括包含bug、重复覆盖范围，或是因过度模拟而无法发现真正回归问题的测试。

Process

流程

At a high level, you'll follow the steps below, then cross-reference load-bearing code with test imports. Importance can be ranked by production blast radius (which code is most depended-on — if you use gjalla, impact and change history surface this), recent bug history, and how deterministic the failure mode is.

从整体层面来说，你需要遵循以下步骤，然后将核心代码与测试导入进行交叉比对。重要性可根据生产影响范围（哪些代码依赖度最高——如果你使用gjalla，可通过影响范围和变更历史查看）、近期bug历史以及故障模式的确定性来排序。

Phase 1: Orient

第一阶段：熟悉情况

Understand the project's test infrastructure before diving in.

Map test structure: Find all test directories, count files per directory, identify naming conventions (
```
.test.ts
```
,
```
.pglite.test.ts
```
,
```
.integration.test.ts
```
, etc.)
Identify test layers: Which tests use real databases (PGlite, SQLite)? Which mock the ORM? Which mock at the service boundary? Which use
```
@vitest-environment node
```
vs
```
jsdom
```
?
Identify critical components: What are the security boundaries, data access layers, and core business logic? These are where false confidence is most dangerous.

在深入之前，先了解项目的测试基础设施。

梳理测试结构：找到所有测试目录，统计每个目录下的文件数量，识别命名规范（如
```
.test.ts
```
、
```
.pglite.test.ts
```
、
```
.integration.test.ts
```
等）
识别测试层级：哪些测试使用真实数据库（PGlite、SQLite）？哪些模拟了ORM？哪些在服务边界进行模拟？哪些使用
```
@vitest-environment node
```
而非
```
jsdom
```
？
识别关键组件：安全边界、数据访问层和核心业务逻辑是什么？这些地方出现虚假信心的风险最高。

Phase 2: Hunt for anti-patterns

第二阶段：排查反模式

Launch parallel investigations across test layers. For each test file, read BOTH the test AND the source code it claims to test. The anti-patterns to find:

在各测试层级同步开展排查。对于每个测试文件，要同时阅读测试代码及其声称要测试的源代码。需要排查的反模式包括：

Anti-pattern 1: Reimplemented logic tests

反模式1：重实现逻辑测试

Tests that never import the real code. Instead they redefine the logic inline and test their own copy. Signals:

```
@vitest-environment node
```
with no component/hook imports
Local functions named
```
simulate*
```
or
```
handle*
```
that mirror source code
Test file has zero imports from
```
src/
```
or source directories

These tests will NEVER catch a regression because they don't exercise the real code.

从未导入真实代码的测试，而是在内部重新定义逻辑并测试自己的副本。特征：

使用
```
@vitest-environment node
```
但未导入组件/钩子
存在名为
```
simulate*
```
或
```
handle*
```
的本地函数，与源代码逻辑镜像
测试文件从未从
```
src/
```
或源目录导入任何内容

这类测试永远无法发现回归问题，因为它们根本没有运行真实代码。

Anti-pattern 2: Tautological mock tests

反模式2：同义反复的模拟测试

Tests that mock the entire database/ORM chain with hardcoded returns, then assert those same hardcoded values. Signals:

mockReturnValue

mockResolvedValue

db.select().from().where()

chains

Queue-based mock infrastructure (
```
_setSelectQueue
```
,
```
pushSelectResult
```
)
Assertions like
```
expect(result).toEqual(mockReturnValue)
```
where
```
mockReturnValue
```
is what the mock was set up to return
Builder pattern mocks where
```
.from()
```
,
```
.where()
```
,
```
.innerJoin()
```
all ignore their arguments

Key test: Could a bug in the real code (wrong table, wrong column, wrong WHERE clause, wrong JOIN) cause this test to fail? If no, the test is tautological.

将整个数据库/ORM调用链用硬编码返回值模拟，然后断言这些相同的硬编码值。特征：

在

db.select().from().where()

调用链上使用

mockReturnValue

mockResolvedValue

基于队列的模拟基础设施（如
```
_setSelectQueue
```
、
```
pushSelectResult
```
）

类似

expect(result).toEqual(mockReturnValue)

的断言，其中

mockReturnValue

正是模拟设置的返回值

构建器模式模拟中，
```
.from()
```
、
```
.where()
```
、
```
.innerJoin()
```
完全忽略传入的参数

关键测试：真实代码中的bug（如错误的表、错误的列、错误的WHERE子句、错误的JOIN）会导致这个测试失败吗？如果不会，那这个测试就是同义反复的。

Anti-pattern 3: Tests encoding wrong behavior

反模式3：包含错误行为的测试

Tests whose assertions verify incorrect behavior that happens to match buggy source code. Signals:

Test fixtures using field names that don't match the source (e.g., test uses
```
assignedTier
```
but source reads
```
subscriptionTier
```
)
Inconsistent thresholds between services tested independently
Mock return values that paper over logic the test claims to verify
Test names that say one thing but assert another (e.g., "returns 403" but asserts
```
toBe(404)
```
)

Key test: Does the test's mock data match what real upstream code actually produces? Or was it hand-crafted to match the (possibly buggy) function under test?

断言验证错误行为的测试，而这些错误行为恰好与有bug的源代码匹配。特征：

测试 fixture 使用的字段名与源代码不匹配（例如测试用
```
assignedTier
```
但源代码读取
```
subscriptionTier
```
）
独立测试的服务之间阈值不一致
模拟返回值掩盖了测试声称要验证的逻辑
测试名称描述的是一件事，但断言的是另一件事（例如名称是“返回403”但断言
```
toBe(404)
```
）

关键测试：测试的模拟数据是否与真实上游代码实际生成的数据一致？还是为了匹配（可能存在bug的）被测函数而手工编写的？

Anti-pattern 4: Redundant companion tests

反模式4：冗余配套测试

Tests that are fully covered by a more rigorous companion file. Signals:

A
```
.test.ts
```
file that mocks the DB alongside a
```
.pglite.test.ts
```
file that tests real SQL for the same class
An "integration" test that mocks at the same level as the "unit" test
Multiple test files for the same source file with overlapping
```
describe
```
/
```
it
```
blocks

完全被更严谨的配套文件覆盖的测试。特征：

一个
```
.test.ts
```
文件模拟数据库，同时存在一个
```
.pglite.test.ts
```
文件针对同一类测试真实SQL
“集成”测试与“单元”测试在同一层级进行模拟
同一源文件对应多个测试文件，且
```
describe
```
/
```
it
```
块重叠

Anti-pattern 5: Placeholder tests

反模式5：占位测试

```
expect(true).toBe(true)
```
Tests with descriptive names but no real assertions
Tests that call a mock and then assert the mock was called (tautology)

```
expect(true).toBe(true)
```
测试名称描述性强但没有实际断言
调用模拟后断言模拟被调用（同义反复）

Phase 3: Classify findings

第三阶段：分类发现的问题

Organize findings into tiers:

Tier	Description	Action
Tier 1	Tests that exercise zero real code (reimplemented logic, inline mock handlers)	Delete entire file
Tier 2	Files with mixed useful and tautological tests	Delete tautological sections, keep logic tests
Tier 3	Tautological tests that have a real companion (PGlite, integration)	Delete redundant mocked version
Tier 4	Tautological tests with NO real companion	Flag as dangerous false confidence. These need real tests written.
Bugs	Tests that encode wrong behavior in source code	Fix source code AND test

将发现的问题分为不同层级：

层级	描述	操作
层级1	完全不运行真实代码的测试（重实现逻辑、内联模拟处理程序）	删除整个文件
层级2	混合了有用测试和同义反复测试的文件	删除同义反复部分，保留逻辑测试
层级3	有真实配套测试（PGlite、集成测试）的同义反复测试	删除冗余的模拟版本
层级4	没有真实配套测试的同义反复测试	标记为危险的虚假信心测试，需要编写真实测试
Bug	源代码中包含错误行为的测试	修复源代码和测试

Phase 4: Report

第四阶段：报告

Present findings as a structured report with:

Tests encoding bugs (highest priority) - these are masking real production issues
Tier 1-3 deletions with file paths and line counts
Tier 4 gaps - areas where coverage will honestly drop and needs real tests
Impact analysis: what gets more robust, what bugs may surface, coverage impact

将发现的问题整理为结构化报告，包含：

包含bug的测试（最高优先级）——这些测试掩盖了真实的生产问题
层级1-3待删除内容，包含文件路径和行数
层级4缺口——测试覆盖范围会真实下降的区域，需要编写真实测试
影响分析：哪些部分会更健壮，可能会暴露哪些bug，对覆盖范围的影响

Phase 5: Clean up (if approved)

第五阶段：清理（如获批准）

Execute in waves, running tests between each:

Delete Tier 1 files (zero real code tested)
Remove Tier 2 tautological sections
Remove Tier 3 redundant tests (where companion exists)
Fix tests encoding wrong behavior + fix source bugs
Run coverage to identify honest gaps
Write real tests for Tier 4 gaps (PGlite for DB-dependent code)

分阶段执行，每阶段后运行测试：

删除层级1文件（完全未测试真实代码）
移除层级2中的同义反复部分
移除层级3中的冗余测试（存在配套测试的情况）
修复包含错误行为的测试 + 修复源代码bug
运行覆盖测试以识别真实缺口
为层级4缺口编写真实测试（针对依赖数据库的代码使用PGlite）

Principles

原则

A test that can't fail is worse than no test. It inflates confidence without providing protection.
Mock at boundaries, not internals. Mock external services (Supabase, Stripe, GitHub API). Don't mock your own database layer when you can use PGlite.
Round-trip tests catch key mismatches. When function A writes data that function B reads, test them together using A's actual output as B's input. Hand-crafted fixtures can silently encode bugs.
Coverage numbers lie when tests are tautological. A line is not "covered" if the mock intercepted every meaningful operation. Honest coverage comes from tests that exercise real code paths.
Delete before you write. Remove false confidence first so the coverage report becomes an honest map of where you're strong and where you're exposed. Then fill gaps.

无法失败的测试比没有测试更糟。它会夸大信心却无法提供保护。
在边界处模拟，而非内部。模拟外部服务（Supabase、Stripe、GitHub API）。当可以使用PGlite时，不要模拟自己的数据库层。
往返测试能发现关键不匹配。当函数A写入的数据被函数B读取时，将它们放在一起测试，用A的实际输出作为B的输入。手工编写的fixture可能会悄悄包含bug。
当测试是同义反复时，覆盖范围数值不可信。如果模拟拦截了所有有意义的操作，那么代码行并非真正“被覆盖”。真实的覆盖范围来自运行真实代码路径的测试。
先删除再编写。首先移除虚假信心测试，让覆盖报告真实反映你的优势和薄弱区域，然后再填补缺口。