fixing-flaky-tests

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Fixing Flaky Tests

修复不稳定测试

Target symptom: Tests pass when run alone, fail when run with other tests.

目标症状： 测试单独运行时通过，与其他测试一起运行时失败。

Diagnose first

先进行诊断

Test passes alone, fails with others?
    │
    ├─ Same error every time → Shared state
    │   └─ Database, globals, files, singletons
    │
    ├─ Random/timing failures → Race condition
    │   └─ Use `condition-based-waiting` skill
    │
    └─ Resource errors (port, file lock) → Resource conflict
        └─ Need unique resources per test/worker

Quick diagnosis:

Run failing test 10x alone - does it always pass?
Run failing test 10x with the suite - same error or different?
Check error message - mentions port/file/connection?

测试单独运行通过，和其他测试一起运行失败？
    │
    ├─ 每次错误都相同 → 共享状态问题
    │   └─ 数据库、全局变量、文件、单例
    │
    ├─ 随机/时序相关失败 → 竞态条件
    │   └─ 使用`condition-based-waiting`技能
    │
    └─ 资源错误（端口、文件锁）→ 资源冲突
        └─ 为每个测试/工作进程分配唯一资源

快速诊断步骤：

单独运行失败的测试10次 - 是否每次都通过？
在测试套件中运行失败的测试10次 - 错误相同还是不同？
查看错误信息 - 是否提到端口/文件/连接？

Shared state (deterministic failures)

共享状态问题（确定性失败）

Tests pollute state that other tests depend on. Fix by isolating state per test.

State Type	Isolation Pattern
Database	Transaction rollback, savepoints, worker-specific DBs
Global variables	Reset in `beforeEach` / `afterEach`
Singletons	Provide fresh instance per test
Module state	`jest.resetModules()` or equivalent
Files	Unique paths per test, temp directories
Environment vars	Save/restore in setup/teardown

Database isolation (most common):

python

undefined

测试污染了其他测试依赖的状态。通过为每个测试隔离状态来修复。

状态类型	隔离模式
Database	事务回滚、保存点、每个工作进程独立数据库
全局变量	在 `beforeEach` / `afterEach` 中重置
单例	为每个测试提供全新实例
模块状态	使用 `jest.resetModules()` 或等效方法
文件	为每个测试分配唯一路径、使用临时目录
环境变量	在设置/清理阶段保存/恢复

数据库隔离（最常见场景）：

python

undefined

Python: Savepoint rollback - each test gets rolled back

Python: 保存点回滚 - 每个测试的修改都会被回滚

@pytest.fixture async def db_session(db_engine): async with db_engine.connect() as conn: await conn.begin() await conn.begin_nested() # Savepoint # ... yield session ... await conn.rollback() # All changes vanish


```typescript
// Jest: Reset mocks between tests
beforeEach(() => {
  jest.clearAllMocks()
  jest.resetModules()  // Clear module cache before test
})

afterEach(() => {
  jest.restoreAllMocks()  // Restore spied functions
})

See language-specific references for complete patterns.

@pytest.fixture async def db_session(db_engine): async with db_engine.connect() as conn: await conn.begin() await conn.begin_nested() # 保存点 # ... 传递session ... await conn.rollback() # 所有修改都会被撤销


```typescript
// Jest: 在测试之间重置模拟
beforeEach(() => {
  jest.clearAllMocks()
  jest.resetModules()  // 在测试前清除模块缓存
})

afterEach(() => {
  jest.restoreAllMocks()  // 恢复被监听的函数
})

查看特定语言的参考文档获取完整实现模式。

Race conditions (random failures)

竞态条件（随机失败）

Tests don't wait for async operations to complete.

Use the
condition-based-waiting
skill for detailed patterns on:

Framework-specific waiting (Testing Library
```
findBy
```
, Playwright auto-wait)
Custom polling helpers
When arbitrary timeouts are acceptable

Quick summary: Wait for conditions, not time:

typescript

// Bad
await sleep(500)

// Good
await waitFor(() => expect(result).toBe('done'))

测试未等待异步操作完成。

使用
condition-based-waiting
技能获取以下场景的详细实现模式：

框架专属等待方法（Testing Library
```
findBy
```
、Playwright自动等待）
自定义轮询工具
何时可以使用任意超时

快速总结：等待条件，而非固定时间：

typescript

// 不良实践
await sleep(500)

// 最佳实践
await waitFor(() => expect(result).toBe('done'))

Resource conflicts (port/file errors)

资源冲突（端口/文件错误）

Multiple tests or workers compete for same resource.

Worker-specific resources:

python

undefined

多个测试或工作进程竞争同一资源。

为每个工作进程分配专属资源：

python

undefined

Python pytest-xdist: unique DB per worker

Python pytest-xdist: 每个工作进程使用独立数据库

@pytest.fixture(scope="session") def database_url(worker_id): if worker_id == "master": return "postgresql://localhost/test" return f"postgresql://localhost/test_{worker_id}"


```typescript
// Jest/Node: dynamic port allocation
const server = app.listen(0)  // OS assigns available port
const port = server.address().port

File conflicts:

python

import tempfile

@pytest.fixture
def temp_dir():
    with tempfile.TemporaryDirectory() as d:
        yield d

@pytest.fixture(scope="session") def database_url(worker_id): if worker_id == "master": return "postgresql://localhost/test" return f"postgresql://localhost/test_{worker_id}"


```typescript
// Jest/Node: 动态端口分配
const server = app.listen(0)  // 由系统分配可用端口
const port = server.address().port

文件冲突处理：

python

import tempfile

@pytest.fixture
def temp_dir():
    with tempfile.TemporaryDirectory() as d:
        yield d

Language-specific isolation patterns

特定语言的隔离模式

Stack	Reference
Python (pytest, SQLAlchemy)	references/python.md
Jest / Testing Library	references/jest.md
Playwright E2E	references/playwright.md

技术栈	参考文档
Python (pytest, SQLAlchemy)	references/python.md
Jest / Testing Library	references/jest.md
Playwright E2E	references/playwright.md

Verification

验证修复效果

After fixing, verify the fix worked:

bash

undefined

修复完成后，验证修复是否有效：

bash

undefined

Run the specific test many times

多次运行特定测试

pytest tests/test_flaky.py -x --count=20

Run with parallelism

并行运行测试

pytest -n auto

Jest equivalent

Jest 等效命令

jest --runInBand # First verify serial works jest # Then verify parallel works

undefined

jest --runInBand # 先验证串行运行正常 jest # 再验证并行运行正常

undefined