test-automation-strategy

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Test Automation Strategy

测试自动化策略

<default_to_action> When designing or improving test automation:
  1. FOLLOW test pyramid: 70% unit, 20% integration, 10% E2E
  2. APPLY F.I.R.S.T. principles: Fast, Isolated, Repeatable, Self-validating, Timely
  3. USE patterns: Page Object Model, Builder pattern, Factory pattern
  4. INTEGRATE in CI/CD: Every commit runs tests, fail fast, clear feedback
  5. MANAGE flaky tests: Quarantine, fix, or delete - never ignore
Quick Anti-Pattern Detection:
  • Ice cream cone (many E2E, few unit) → Invert to pyramid
  • Slow tests (> 10 min suite) → Parallelize, mock external deps
  • Flaky tests → Fix timing, isolate data, or quarantine
  • Test duplication → Share fixtures, use page objects
  • Brittle selectors → Use data-testid, semantic locators
Critical Success Factors:
  • Fast feedback is the goal (< 10 min full suite)
  • Automation supports testing, doesn't replace judgment
  • Invest in test infrastructure like production code </default_to_action>
<default_to_action> 在设计或优化测试自动化时:
  1. 遵循测试金字塔原则:70% 单元测试、20% 集成测试、10% 端到端(E2E)测试
  2. 应用F.I.R.S.T.原则:快速(Fast)、独立(Isolated)、可重复(Repeatable)、自验证(Self-validating)、及时(Timely)
  3. 使用设计模式:页面对象模型(Page Object Model)、建造者模式(Builder pattern)、工厂模式(Factory pattern)
  4. 集成到CI/CD流程:每次提交都运行测试、快速失败、提供清晰反馈
  5. 管理不稳定测试:隔离、修复或删除——绝不要忽略
快速识别反模式:
  • 冰淇淋锥型(大量E2E测试,少量单元测试)→ 调整为金字塔结构
  • 测试套件运行缓慢(超过10分钟)→ 并行执行、模拟外部依赖
  • 不稳定测试 → 修复时序问题、隔离数据或进行隔离处理
  • 测试重复 → 共享测试夹具、使用页面对象
  • 脆弱选择器 → 使用data-testid、语义化定位器
关键成功因素:
  • 核心目标是快速反馈(完整套件运行时间<10分钟)
  • 自动化是测试的辅助手段,无法替代人工判断
  • 像投入生产代码一样投入测试基础设施建设 </default_to_action>

Quick Reference Card

快速参考卡片

When to Use

适用场景

  • Building new automation framework
  • Improving existing test efficiency
  • Reducing flaky test burden
  • Optimizing CI/CD pipeline speed
  • 构建新的自动化框架
  • 提升现有测试效率
  • 减轻不稳定测试带来的负担
  • 优化CI/CD流水线速度

Test Pyramid

测试金字塔

Layer%SpeedIsolationExamples
Unit70%< 1msCompletePure functions, logic
Integration20%< 1sPartialAPI, database
E2E10%< 30sNoneUser journeys
层级占比速度隔离性示例
Unit70%< 1ms完全隔离纯函数、业务逻辑
Integration20%< 1s部分隔离API、数据库
E2E10%< 30s无隔离用户流程

F.I.R.S.T. Principles

F.I.R.S.T.原则

PrincipleMeaningHow
FastQuick executionMock external deps
IsolatedNo shared stateFresh fixtures per test
RepeatableSame result every timeNo random data
Self-validatingClear pass/failAssert, don't print
TimelyWritten with codeTDD, not after
原则含义实施方式
Fast执行速度快模拟外部依赖
Isolated无共享状态每个测试使用全新夹具
Repeatable结果一致不使用随机数据
Self-validating清晰的通过/失败结果使用断言,而非打印输出
Timely与代码同步编写采用测试驱动开发(TDD),而非事后补充

Anti-Patterns

反模式

ProblemSymptomFix
Ice cream cone80% E2E, 10% unitInvert pyramid
Slow suite30+ min CIParallelize, prune
Flaky testsRandom failuresQuarantine, fix timing
Coupled testsOrder-dependentIsolate data
Brittle selectorsBreak on CSS changeUse data-testid

问题症状修复方案
冰淇淋锥型80%为E2E测试,10%为单元测试调整为金字塔结构
测试套件缓慢CI流水线运行时间超过30分钟并行执行、精简测试
不稳定测试随机失败隔离处理、修复时序问题
测试耦合测试依赖执行顺序隔离测试数据
脆弱选择器CSS变更导致测试失败使用data-testid

Page Object Model

Page Object Model

javascript
// pages/LoginPage.js
class LoginPage {
  constructor(page) {
    this.page = page;
    this.emailInput = '[data-testid="email"]';
    this.passwordInput = '[data-testid="password"]';
    this.submitButton = '[data-testid="submit"]';
    this.errorMessage = '[data-testid="error"]';
  }

  async login(email, password) {
    await this.page.fill(this.emailInput, email);
    await this.page.fill(this.passwordInput, password);
    await this.page.click(this.submitButton);
  }

  async getError() {
    return this.page.textContent(this.errorMessage);
  }
}

// Test uses page object
test('shows error for invalid credentials', async ({ page }) => {
  const loginPage = new LoginPage(page);
  await loginPage.login('bad@email.com', 'wrong');
  expect(await loginPage.getError()).toBe('Invalid credentials');
});

javascript
// pages/LoginPage.js
class LoginPage {
  constructor(page) {
    this.page = page;
    this.emailInput = '[data-testid="email"]';
    this.passwordInput = '[data-testid="password"]';
    this.submitButton = '[data-testid="submit"]';
    this.errorMessage = '[data-testid="error"]';
  }

  async login(email, password) {
    await this.page.fill(this.emailInput, email);
    await this.page.fill(this.passwordInput, password);
    await this.page.click(this.submitButton);
  }

  async getError() {
    return this.page.textContent(this.errorMessage);
  }
}

// Test uses page object
test('shows error for invalid credentials', async ({ page }) => {
  const loginPage = new LoginPage(page);
  await loginPage.login('bad@email.com', 'wrong');
  expect(await loginPage.getError()).toBe('Invalid credentials');
});

CI/CD Integration

CI/CD Integration

yaml
name: Test Pipeline
on: [push, pull_request]

jobs:
  unit-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npm run test:unit -- --coverage
        timeout-minutes: 5
      - uses: codecov/codecov-action@v3

  integration-tests:
    needs: unit-tests
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:15
    steps:
      - run: npm run test:integration
        timeout-minutes: 10

  e2e-tests:
    needs: integration-tests
    runs-on: ubuntu-latest
    steps:
      - run: npx playwright test
        timeout-minutes: 15

yaml
name: Test Pipeline
on: [push, pull_request]

jobs:
  unit-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npm run test:unit -- --coverage
        timeout-minutes: 5
      - uses: codecov/codecov-action@v3

  integration-tests:
    needs: unit-tests
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:15
    steps:
      - run: npm run test:integration
        timeout-minutes: 10

  e2e-tests:
    needs: integration-tests
    runs-on: ubuntu-latest
    steps:
      - run: npx playwright test
        timeout-minutes: 15

Flaky Test Management

Flaky Test Management

typescript
// Quarantine flaky tests
describe.skip('Quarantined - INC-123', () => {
  test('flaky test awaiting fix', () => { /* ... */ });
});

// Agent-assisted stabilization
await Task("Fix Flaky Tests", {
  tests: quarantinedTests,
  analysis: ['timing-issues', 'data-isolation', 'race-conditions'],
  strategies: ['add-waits', 'isolate-fixtures', 'mock-externals']
}, "qe-flaky-test-hunter");

typescript
// Quarantine flaky tests
describe.skip('Quarantined - INC-123', () => {
  test('flaky test awaiting fix', () => { /* ... */ });
});

// Agent-assisted stabilization
await Task("Fix Flaky Tests", {
  tests: quarantinedTests,
  analysis: ['timing-issues', 'data-isolation', 'race-conditions'],
  strategies: ['add-waits', 'isolate-fixtures', 'mock-externals']
}, "qe-flaky-test-hunter");

Agent-Assisted Automation

Agent-Assisted Automation

typescript
// Generate tests following pyramid
await Task("Generate Test Suite", {
  sourceCode: 'src/',
  pyramid: { unit: 70, integration: 20, e2e: 10 },
  patterns: ['page-object', 'builder', 'factory'],
  framework: 'jest'
}, "qe-test-generator");

// Optimize test execution
await Task("Optimize Suite", {
  algorithm: 'johnson-lindenstrauss',
  targetReduction: 0.3,
  maintainCoverage: 0.95
}, "qe-regression-risk-analyzer");

// Analyze flaky patterns
await Task("Flaky Analysis", {
  testHistory: 'last-30-days',
  detectPatterns: ['timing', 'data', 'environment'],
  recommend: 'stabilization-strategy'
}, "qe-flaky-test-hunter");

typescript
// Generate tests following pyramid
await Task("Generate Test Suite", {
  sourceCode: 'src/',
  pyramid: { unit: 70, integration: 20, e2e: 10 },
  patterns: ['page-object', 'builder', 'factory'],
  framework: 'jest'
}, "qe-test-generator");

// Optimize test execution
await Task("Optimize Suite", {
  algorithm: 'johnson-lindenstrauss',
  targetReduction: 0.3,
  maintainCoverage: 0.95
}, "qe-regression-risk-analyzer");

// Analyze flaky patterns
await Task("Flaky Analysis", {
  testHistory: 'last-30-days',
  detectPatterns: ['timing', 'data', 'environment'],
  recommend: 'stabilization-strategy'
}, "qe-flaky-test-hunter");

Agent Coordination Hints

Agent Coordination Hints

Memory Namespace

Memory Namespace

aqe/automation/
├── test-pyramid/*        - Coverage by layer
├── page-objects/*        - Shared page objects
├── flaky-registry/*      - Quarantined tests
└── execution-metrics/*   - Suite performance data
aqe/automation/
├── test-pyramid/*        - Coverage by layer
├── page-objects/*        - Shared page objects
├── flaky-registry/*      - Quarantined tests
└── execution-metrics/*   - Suite performance data

Fleet Coordination

Fleet Coordination

typescript
const automationFleet = await FleetManager.coordinate({
  strategy: 'test-automation',
  agents: [
    'qe-test-generator',         // Generate pyramid-compliant tests
    'qe-test-executor',          // Parallel execution
    'qe-coverage-analyzer',      // Coverage gaps
    'qe-flaky-test-hunter',      // Flaky detection
    'qe-regression-risk-analyzer' // Smart selection
  ],
  topology: 'hierarchical'
});

typescript
const automationFleet = await FleetManager.coordinate({
  strategy: 'test-automation',
  agents: [
    'qe-test-generator',         // Generate pyramid-compliant tests
    'qe-test-executor',          // Parallel execution
    'qe-coverage-analyzer',      // Coverage gaps
    'qe-flaky-test-hunter',      // Flaky detection
    'qe-regression-risk-analyzer' // Smart selection
  ],
  topology: 'hierarchical'
});

Related Skills

相关技能

  • tdd-london-chicago - TDD for unit tests
  • api-testing-patterns - Integration patterns
  • cicd-pipeline-qe-orchestrator - Pipeline integration
  • shift-left-testing - Early automation

  • tdd-london-chicago - 单元测试的测试驱动开发(TDD)
  • api-testing-patterns - 集成测试模式
  • cicd-pipeline-qe-orchestrator - 流水线集成
  • shift-left-testing - 早期自动化测试

Remember

要点回顾

Pyramid: 70% unit, 20% integration, 10% E2E. F.I.R.S.T. principles for every test. Page Object Model for E2E. Parallelize for speed. Quarantine flaky tests - never ignore them. Treat test code like production code.
With Agents: Agents generate pyramid-compliant tests, detect flaky patterns, optimize execution time, and maintain test infrastructure. Use agents to scale automation quality.
金字塔原则:70%单元测试、20%集成测试、10%E2E测试。 所有测试遵循F.I.R.S.T.原则。E2E测试采用Page Object Model。通过并行执行提升速度。隔离不稳定测试——绝不要忽略。像对待生产代码一样对待测试代码。
借助Agent: Agent可生成符合金字塔原则的测试、检测不稳定测试模式、优化执行时间并维护测试基础设施。使用Agent提升自动化测试的质量与规模。