Back to Details

mutation-testing

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Mutation Testing

Mutation Testing

<default_to_action> When validating test quality or improving test effectiveness:
  1. MUTATE code (change + to -, >= to >, remove statements)
  2. RUN tests against each mutant
  3. VERIFY tests catch mutations (kill mutants)
  4. IDENTIFY surviving mutants (tests need improvement)
  5. STRENGTHEN tests to kill surviving mutants
Quick Mutation Metrics:
  • Mutation Score = Killed / (Killed + Survived)
  • Target: > 80% mutation score
  • Surviving mutants = weak tests
Critical Success Factors:
  • High coverage ≠ good tests (100% coverage, 0% assertions)
  • Mutation testing proves tests actually catch bugs
  • Focus on critical code paths first </default_to_action>
<default_to_action> 在验证测试质量或提升测试有效性时：
  1. 对代码进行MUTATE（突变）操作（将+改为-，>=改为>，移除语句等）
  2. 针对每个突变体运行测试
  3. 验证测试是否能发现突变（杀死突变体）
  4. 识别存活的突变体（对应的测试需要优化）
  5. 强化测试以杀死存活的突变体
快速Mutation指标：
  • Mutation Score（突变分数）= 被杀死的突变体数 /（被杀死的突变体数 + 存活的突变体数）
  • 目标：Mutation Score > 80%
  • 存活的突变体 = 薄弱的测试
关键成功要素：
  • 高覆盖率 ≠ 优质测试（比如100%覆盖率但0%断言）
  • Mutation Testing可证明测试确实能发现漏洞
  • 优先关注关键代码路径 </default_to_action>

Quick Reference Card

快速参考卡片

When to Use

适用场景

  • Evaluating test suite quality
  • Finding gaps in test assertions
  • Proving tests catch bugs
  • Before critical releases
  • 评估测试套件质量
  • 发现测试断言中的漏洞
  • 证明测试能发现漏洞
  • 关键版本发布前

Mutation Score Interpretation

Mutation Score 解读

ScoreInterpretation
90%+Excellent test quality
80-90%Good, minor improvements
60-80%Needs attention
< 60%Significant gaps
分数解读
90%+测试质量极佳
80-90%良好，需小幅优化
60-80%需要重点关注
< 60%存在重大漏洞

Common Mutation Operators

常见Mutation操作符

CategoryOriginalMutant
Arithmetic
a + b
a - b
Relational
x >= 18
x > 18
Logical
a && b
a || b
Conditional
if (x)
if (true)
Statement
return x
(removed)
类别原始代码突变体
算术运算
a + b
a - b
关系运算
x >= 18
x > 18
逻辑运算
a && b
`a
条件判断
if (x)
if (true)
语句修改
return x
(已移除)

How Mutation Testing Works

Mutation Testing的工作原理

javascript
// Original code
function isAdult(age) {
  return age >= 18; // ← Mutant: change >= to >
}

// Strong test (catches mutation)
test('18 is adult', () => {
  expect(isAdult(18)).toBe(true); // Kills mutant!
});

// Weak test (mutation survives)
test('19 is adult', () => {
  expect(isAdult(19)).toBe(true); // Doesn't catch >= vs >
});
// Surviving mutant → Test needs boundary value
javascript
// 原始代码
function isAdult(age) {
  return age >= 18; // ← 突变体：将 >= 改为 >
}

// 强测试（可发现突变）
test('18 is adult', () => {
  expect(isAdult(18)).toBe(true); // 杀死突变体！
});

// 弱测试（突变体存活）
test('19 is adult', () => {
  expect(isAdult(19)).toBe(true); // 无法区分 >= 和 >
});
// 存活的突变体 → 测试需要添加边界值用例

Using Stryker

使用Stryker

bash
undefined
bash
undefined

Install

安装

npm install --save-dev @stryker-mutator/core @stryker-mutator/jest-runner
npm install --save-dev @stryker-mutator/core @stryker-mutator/jest-runner

Initialize

初始化

npx stryker init

**Configuration:**
```json
{
  "packageManager": "npm",
  "reporters": ["html", "clear-text", "progress"],
  "testRunner": "jest",
  "coverageAnalysis": "perTest",
  "mutate": [
    "src/**/*.ts",
    "!src/**/*.spec.ts"
  ],
  "thresholds": {
    "high": 90,
    "low": 70,
    "break": 60
  }
}
Run:
bash
npx stryker run
Output:
Mutation Score: 87.3%
Killed: 124
Survived: 18
No Coverage: 3
Timeout: 1
npx stryker init

**配置：**
```json
{
  "packageManager": "npm",
  "reporters": ["html", "clear-text", "progress"],
  "testRunner": "jest",
  "coverageAnalysis": "perTest",
  "mutate": [
    "src/**/*.ts",
    "!src/**/*.spec.ts"
  ],
  "thresholds": {
    "high": 90,
    "low": 70,
    "break": 60
  }
}
运行：
bash
npx stryker run
输出：
Mutation Score: 87.3%
Killed: 124
Survived: 18
No Coverage: 3
Timeout: 1

Fixing Surviving Mutants

修复存活的突变体

javascript
// Surviving mutant: >= changed to >
function calculateDiscount(quantity) {
  if (quantity >= 10) { // Mutant survives!
    return 0.1;
  }
  return 0;
}

// Original weak test
test('large order gets discount', () => {
  expect(calculateDiscount(15)).toBe(0.1); // Doesn't test boundary
});

// Fixed: Add boundary test
test('exactly 10 gets discount', () => {
  expect(calculateDiscount(10)).toBe(0.1); // Kills mutant!
});

test('9 does not get discount', () => {
  expect(calculateDiscount(9)).toBe(0); // Tests below boundary
});
javascript
// 存活的突变体：>= 被改为 >
function calculateDiscount(quantity) {
  if (quantity >= 10) { // 突变体存活！
    return 0.1;
  }
  return 0;
}

// 原始的弱测试
test('large order gets discount', () => {
  expect(calculateDiscount(15)).toBe(0.1); // 未测试边界值
});

// 修复：添加边界测试
test('exactly 10 gets discount', () => {
  expect(calculateDiscount(10)).toBe(0.1); // 杀死突变体！
});

test('9 does not get discount', () => {
  expect(calculateDiscount(9)).toBe(0); // 测试边界值以下的情况
});

Agent-Driven Mutation Testing

基于Agent的Mutation Testing

typescript
// Analyze mutation score and generate fixes
await Task("Mutation Analysis", {
  targetFile: 'src/payment.ts',
  generateMissingTests: true,
  minScore: 80
}, "qe-test-generator");

// Returns:
// {
//   mutationScore: 0.65,
//   survivedMutations: [
//     { line: 45, operator: '>=', mutant: '>', killedBy: null }
//   ],
//   generatedTests: [
//     'test for boundary at line 45'
//   ]
// }

// Coverage + mutation correlation
await Task("Coverage Quality Analysis", {
  coverageData: coverageReport,
  mutationData: mutationReport,
  identifyWeakCoverage: true
}, "qe-coverage-analyzer");
typescript
// 分析Mutation Score并生成修复方案
await Task("Mutation Analysis", {
  targetFile: 'src/payment.ts',
  generateMissingTests: true,
  minScore: 80
}, "qe-test-generator");

// 返回结果：
// {
//   mutationScore: 0.65,
//   survivedMutations: [
//     { line: 45, operator: '>=', mutant: '>', killedBy: null }
//   ],
//   generatedTests: [
//     'test for boundary at line 45'
//   ]
// }

// 覆盖率与突变的关联分析
await Task("Coverage Quality Analysis", {
  coverageData: coverageReport,
  mutationData: mutationReport,
  identifyWeakCoverage: true
}, "qe-coverage-analyzer");

Agent Coordination Hints

Agent协作提示

Memory Namespace

内存命名空间

aqe/mutation-testing/
├── mutation-results/*   - Stryker reports
├── surviving/*          - Surviving mutants
├── generated-tests/*    - Tests to kill mutants
└── trends/*             - Mutation score over time
aqe/mutation-testing/
├── mutation-results/*   - Stryker报告
├── surviving/*          - 存活的突变体
├── generated-tests/*    - 用于杀死突变体的测试用例
└── trends/*             - Mutation Score变化趋势

Fleet Coordination

集群协作

typescript
const mutationFleet = await FleetManager.coordinate({
  strategy: 'mutation-testing',
  agents: [
    'qe-test-generator',     // Generate tests for survivors
    'qe-coverage-analyzer',  // Coverage correlation
    'qe-quality-analyzer'    // Quality assessment
  ],
  topology: 'sequential'
});
typescript
const mutationFleet = await FleetManager.coordinate({
  strategy: 'mutation-testing',
  agents: [
    'qe-test-generator',     // 为存活突变体生成测试用例
    'qe-coverage-analyzer',  // 覆盖率关联分析
    'qe-quality-analyzer'    // 质量评估
  ],
  topology: 'sequential'
});

Related Skills

相关技能

  • tdd-london-chicago - Write effective tests first
  • test-design-techniques - Boundary value analysis
  • quality-metrics - Measure test effectiveness
  • tdd-london-chicago - 先编写高效的测试用例
  • test-design-techniques - 边界值分析
  • quality-metrics - 评估测试有效性

Remember

注意事项

High code coverage ≠ good tests. 100% coverage but weak assertions = useless. Mutation testing proves tests actually catch bugs.
Focus on critical paths first. Don't mutation test everything - prioritize payment, authentication, data integrity code.
With Agents: Agents run mutation analysis, identify surviving mutants, and generate missing test cases to kill them. Automated improvement of test quality.
高代码覆盖率 ≠ 优质测试。100%覆盖率但断言薄弱的测试毫无用处。Mutation Testing可证明测试确实能发现漏洞。
优先关注关键路径。不要对所有代码都进行突变测试——优先处理支付、认证、数据完整性相关代码。
借助Agent： Agent可运行突变分析、识别存活的突变体，并生成缺失的测试用例来杀死这些突变体，实现测试质量的自动化提升。