test-automation-strategy
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChineseTest Automation Strategy
测试自动化策略
<default_to_action>
When designing or improving test automation:
- FOLLOW test pyramid: 70% unit, 20% integration, 10% E2E
- APPLY F.I.R.S.T. principles: Fast, Isolated, Repeatable, Self-validating, Timely
- USE patterns: Page Object Model, Builder pattern, Factory pattern
- INTEGRATE in CI/CD: Every commit runs tests, fail fast, clear feedback
- MANAGE flaky tests: Quarantine, fix, or delete - never ignore
Quick Anti-Pattern Detection:
- Ice cream cone (many E2E, few unit) → Invert to pyramid
- Slow tests (> 10 min suite) → Parallelize, mock external deps
- Flaky tests → Fix timing, isolate data, or quarantine
- Test duplication → Share fixtures, use page objects
- Brittle selectors → Use data-testid, semantic locators
Critical Success Factors:
- Fast feedback is the goal (< 10 min full suite)
- Automation supports testing, doesn't replace judgment
- Invest in test infrastructure like production code </default_to_action>
<default_to_action>
在设计或优化测试自动化时:
- 遵循测试金字塔原则:70% 单元测试、20% 集成测试、10% 端到端(E2E)测试
- 应用F.I.R.S.T.原则:快速(Fast)、独立(Isolated)、可重复(Repeatable)、自验证(Self-validating)、及时(Timely)
- 使用设计模式:页面对象模型(Page Object Model)、建造者模式(Builder pattern)、工厂模式(Factory pattern)
- 集成到CI/CD流程:每次提交都运行测试、快速失败、提供清晰反馈
- 管理不稳定测试:隔离、修复或删除——绝不要忽略
快速识别反模式:
- 冰淇淋锥型(大量E2E测试,少量单元测试)→ 调整为金字塔结构
- 测试套件运行缓慢(超过10分钟)→ 并行执行、模拟外部依赖
- 不稳定测试 → 修复时序问题、隔离数据或进行隔离处理
- 测试重复 → 共享测试夹具、使用页面对象
- 脆弱选择器 → 使用data-testid、语义化定位器
关键成功因素:
- 核心目标是快速反馈(完整套件运行时间<10分钟)
- 自动化是测试的辅助手段,无法替代人工判断
- 像投入生产代码一样投入测试基础设施建设 </default_to_action>
Quick Reference Card
快速参考卡片
When to Use
适用场景
- Building new automation framework
- Improving existing test efficiency
- Reducing flaky test burden
- Optimizing CI/CD pipeline speed
- 构建新的自动化框架
- 提升现有测试效率
- 减轻不稳定测试带来的负担
- 优化CI/CD流水线速度
Test Pyramid
测试金字塔
| Layer | % | Speed | Isolation | Examples |
|---|---|---|---|---|
| Unit | 70% | < 1ms | Complete | Pure functions, logic |
| Integration | 20% | < 1s | Partial | API, database |
| E2E | 10% | < 30s | None | User journeys |
| 层级 | 占比 | 速度 | 隔离性 | 示例 |
|---|---|---|---|---|
| Unit | 70% | < 1ms | 完全隔离 | 纯函数、业务逻辑 |
| Integration | 20% | < 1s | 部分隔离 | API、数据库 |
| E2E | 10% | < 30s | 无隔离 | 用户流程 |
F.I.R.S.T. Principles
F.I.R.S.T.原则
| Principle | Meaning | How |
|---|---|---|
| Fast | Quick execution | Mock external deps |
| Isolated | No shared state | Fresh fixtures per test |
| Repeatable | Same result every time | No random data |
| Self-validating | Clear pass/fail | Assert, don't print |
| Timely | Written with code | TDD, not after |
| 原则 | 含义 | 实施方式 |
|---|---|---|
| Fast | 执行速度快 | 模拟外部依赖 |
| Isolated | 无共享状态 | 每个测试使用全新夹具 |
| Repeatable | 结果一致 | 不使用随机数据 |
| Self-validating | 清晰的通过/失败结果 | 使用断言,而非打印输出 |
| Timely | 与代码同步编写 | 采用测试驱动开发(TDD),而非事后补充 |
Anti-Patterns
反模式
| Problem | Symptom | Fix |
|---|---|---|
| Ice cream cone | 80% E2E, 10% unit | Invert pyramid |
| Slow suite | 30+ min CI | Parallelize, prune |
| Flaky tests | Random failures | Quarantine, fix timing |
| Coupled tests | Order-dependent | Isolate data |
| Brittle selectors | Break on CSS change | Use data-testid |
| 问题 | 症状 | 修复方案 |
|---|---|---|
| 冰淇淋锥型 | 80%为E2E测试,10%为单元测试 | 调整为金字塔结构 |
| 测试套件缓慢 | CI流水线运行时间超过30分钟 | 并行执行、精简测试 |
| 不稳定测试 | 随机失败 | 隔离处理、修复时序问题 |
| 测试耦合 | 测试依赖执行顺序 | 隔离测试数据 |
| 脆弱选择器 | CSS变更导致测试失败 | 使用data-testid |
Page Object Model
Page Object Model
javascript
// pages/LoginPage.js
class LoginPage {
constructor(page) {
this.page = page;
this.emailInput = '[data-testid="email"]';
this.passwordInput = '[data-testid="password"]';
this.submitButton = '[data-testid="submit"]';
this.errorMessage = '[data-testid="error"]';
}
async login(email, password) {
await this.page.fill(this.emailInput, email);
await this.page.fill(this.passwordInput, password);
await this.page.click(this.submitButton);
}
async getError() {
return this.page.textContent(this.errorMessage);
}
}
// Test uses page object
test('shows error for invalid credentials', async ({ page }) => {
const loginPage = new LoginPage(page);
await loginPage.login('bad@email.com', 'wrong');
expect(await loginPage.getError()).toBe('Invalid credentials');
});javascript
// pages/LoginPage.js
class LoginPage {
constructor(page) {
this.page = page;
this.emailInput = '[data-testid="email"]';
this.passwordInput = '[data-testid="password"]';
this.submitButton = '[data-testid="submit"]';
this.errorMessage = '[data-testid="error"]';
}
async login(email, password) {
await this.page.fill(this.emailInput, email);
await this.page.fill(this.passwordInput, password);
await this.page.click(this.submitButton);
}
async getError() {
return this.page.textContent(this.errorMessage);
}
}
// Test uses page object
test('shows error for invalid credentials', async ({ page }) => {
const loginPage = new LoginPage(page);
await loginPage.login('bad@email.com', 'wrong');
expect(await loginPage.getError()).toBe('Invalid credentials');
});CI/CD Integration
CI/CD Integration
yaml
name: Test Pipeline
on: [push, pull_request]
jobs:
unit-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: npm ci
- run: npm run test:unit -- --coverage
timeout-minutes: 5
- uses: codecov/codecov-action@v3
integration-tests:
needs: unit-tests
runs-on: ubuntu-latest
services:
postgres:
image: postgres:15
steps:
- run: npm run test:integration
timeout-minutes: 10
e2e-tests:
needs: integration-tests
runs-on: ubuntu-latest
steps:
- run: npx playwright test
timeout-minutes: 15yaml
name: Test Pipeline
on: [push, pull_request]
jobs:
unit-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: npm ci
- run: npm run test:unit -- --coverage
timeout-minutes: 5
- uses: codecov/codecov-action@v3
integration-tests:
needs: unit-tests
runs-on: ubuntu-latest
services:
postgres:
image: postgres:15
steps:
- run: npm run test:integration
timeout-minutes: 10
e2e-tests:
needs: integration-tests
runs-on: ubuntu-latest
steps:
- run: npx playwright test
timeout-minutes: 15Flaky Test Management
Flaky Test Management
typescript
// Quarantine flaky tests
describe.skip('Quarantined - INC-123', () => {
test('flaky test awaiting fix', () => { /* ... */ });
});
// Agent-assisted stabilization
await Task("Fix Flaky Tests", {
tests: quarantinedTests,
analysis: ['timing-issues', 'data-isolation', 'race-conditions'],
strategies: ['add-waits', 'isolate-fixtures', 'mock-externals']
}, "qe-flaky-test-hunter");typescript
// Quarantine flaky tests
describe.skip('Quarantined - INC-123', () => {
test('flaky test awaiting fix', () => { /* ... */ });
});
// Agent-assisted stabilization
await Task("Fix Flaky Tests", {
tests: quarantinedTests,
analysis: ['timing-issues', 'data-isolation', 'race-conditions'],
strategies: ['add-waits', 'isolate-fixtures', 'mock-externals']
}, "qe-flaky-test-hunter");Agent-Assisted Automation
Agent-Assisted Automation
typescript
// Generate tests following pyramid
await Task("Generate Test Suite", {
sourceCode: 'src/',
pyramid: { unit: 70, integration: 20, e2e: 10 },
patterns: ['page-object', 'builder', 'factory'],
framework: 'jest'
}, "qe-test-generator");
// Optimize test execution
await Task("Optimize Suite", {
algorithm: 'johnson-lindenstrauss',
targetReduction: 0.3,
maintainCoverage: 0.95
}, "qe-regression-risk-analyzer");
// Analyze flaky patterns
await Task("Flaky Analysis", {
testHistory: 'last-30-days',
detectPatterns: ['timing', 'data', 'environment'],
recommend: 'stabilization-strategy'
}, "qe-flaky-test-hunter");typescript
// Generate tests following pyramid
await Task("Generate Test Suite", {
sourceCode: 'src/',
pyramid: { unit: 70, integration: 20, e2e: 10 },
patterns: ['page-object', 'builder', 'factory'],
framework: 'jest'
}, "qe-test-generator");
// Optimize test execution
await Task("Optimize Suite", {
algorithm: 'johnson-lindenstrauss',
targetReduction: 0.3,
maintainCoverage: 0.95
}, "qe-regression-risk-analyzer");
// Analyze flaky patterns
await Task("Flaky Analysis", {
testHistory: 'last-30-days',
detectPatterns: ['timing', 'data', 'environment'],
recommend: 'stabilization-strategy'
}, "qe-flaky-test-hunter");Agent Coordination Hints
Agent Coordination Hints
Memory Namespace
Memory Namespace
aqe/automation/
├── test-pyramid/* - Coverage by layer
├── page-objects/* - Shared page objects
├── flaky-registry/* - Quarantined tests
└── execution-metrics/* - Suite performance dataaqe/automation/
├── test-pyramid/* - Coverage by layer
├── page-objects/* - Shared page objects
├── flaky-registry/* - Quarantined tests
└── execution-metrics/* - Suite performance dataFleet Coordination
Fleet Coordination
typescript
const automationFleet = await FleetManager.coordinate({
strategy: 'test-automation',
agents: [
'qe-test-generator', // Generate pyramid-compliant tests
'qe-test-executor', // Parallel execution
'qe-coverage-analyzer', // Coverage gaps
'qe-flaky-test-hunter', // Flaky detection
'qe-regression-risk-analyzer' // Smart selection
],
topology: 'hierarchical'
});typescript
const automationFleet = await FleetManager.coordinate({
strategy: 'test-automation',
agents: [
'qe-test-generator', // Generate pyramid-compliant tests
'qe-test-executor', // Parallel execution
'qe-coverage-analyzer', // Coverage gaps
'qe-flaky-test-hunter', // Flaky detection
'qe-regression-risk-analyzer' // Smart selection
],
topology: 'hierarchical'
});Related Skills
相关技能
- tdd-london-chicago - TDD for unit tests
- api-testing-patterns - Integration patterns
- cicd-pipeline-qe-orchestrator - Pipeline integration
- shift-left-testing - Early automation
- tdd-london-chicago - 单元测试的测试驱动开发(TDD)
- api-testing-patterns - 集成测试模式
- cicd-pipeline-qe-orchestrator - 流水线集成
- shift-left-testing - 早期自动化测试
Remember
要点回顾
Pyramid: 70% unit, 20% integration, 10% E2E. F.I.R.S.T. principles for every test. Page Object Model for E2E. Parallelize for speed. Quarantine flaky tests - never ignore them. Treat test code like production code.
With Agents: Agents generate pyramid-compliant tests, detect flaky patterns, optimize execution time, and maintain test infrastructure. Use agents to scale automation quality.
金字塔原则:70%单元测试、20%集成测试、10%E2E测试。 所有测试遵循F.I.R.S.T.原则。E2E测试采用Page Object Model。通过并行执行提升速度。隔离不稳定测试——绝不要忽略。像对待生产代码一样对待测试代码。
借助Agent: Agent可生成符合金字塔原则的测试、检测不稳定测试模式、优化执行时间并维护测试基础设施。使用Agent提升自动化测试的质量与规模。