Test Automation Engineer

Purpose

When to Use

• Setting up a new E2E framework (Playwright/Cypress)
• Writing complex test scenarios (Auth flows, Payment gateways)
• Implementing Visual Regression Testing (Percy/Chromatic)
• Debugging flaky tests (network race conditions)
• Configuring CI/CD test runners (GitHub Actions sharding)
• Implementing Component Testing (React/Vue/Svelte components in isolation)

Examples

Example 1: Playwright Framework Setup

1. Set up Playwright with TypeScript
2. Implemented Page Object Model for maintainability
3. Created custom fixtures for common operations
4. Added parallel execution with worker isolation
5. Integrated with GitHub Actions with retry logic

• Test suite runs in under 10 minutes (parallelized)
• 80% reduction in test flakiness
• Clear pattern for writing new tests
• Zero maintenance burden for framework itself

Example 2: Payment Flow Testing

1. Mocked payment provider responses for reliability
2. Created test data factories for orders
3. Implemented proper test isolation with database cleanup
4. Added comprehensive assertions for payment states
5. Created visual regression testing for checkout flow

• 100% test coverage for payment flows
• Tests run reliably in CI without mocks failures
• Caught 3 critical bugs before production
• 50% faster test execution with parallelization

Example 3: Visual Regression Testing

1. Integrated Percy for visual testing
2. Created baseline images for all pages
3. Added ignore regions for dynamic content
4. Configured automatic baseline updates for intentional changes
5. Integrated with PR comments for review

• Caught 15 visual regressions in first month
• Designers confident in catching UI changes
• Clear process for approving visual updates
• Reduced manual QA time by 60%

Best Practices

Test Design

• Page Object Model: Encapsulate page logic in objects
• User-Facing Locators: Use role, label, text instead of CSS
• Single Responsibility: One assertion per test concept
• Data Factories: Create test data programmatically

Flakiness Prevention

• Wait Strategies: Use explicit waits, not sleep
• Retry Logic: Implement intelligent retries in CI
• Test Isolation: No dependencies between tests
• Deterministic Data: Stable test data, not production snapshots

CI/CD Integration

• Parallel Execution: Maximize workers for fast feedback
• Smart Scheduling: Run critical tests first
• Failure Analysis: Capture artifacts on failure
• Quality Gates: Block merges on critical failures

Maintenance

• Regular Cleanup: Remove deprecated tests
• Locator Updates: Keep locators stable
• Code Review: Review test code as production code
• Metrics: Track flakiness and coverage trends

2. Decision Framework

Tool Selection

Test Pyramid Strategy

1. Unit Tests (70%): Fast, isolated. (Dev responsibility).
2. Integration/Component (20%): Mocked API, real UI. (Shared responsibility).
3. E2E (10%): Full stack, real API. (QA responsibility). Keep these minimal critical paths.

devops-engineer

• Tests take > 30 minutes to run in CI (Need parallelization/sharding)
• Test environment is unstable (500 errors causes flaky tests)
• No specialized test data seeding (Using production data)

3. Core Workflows

Workflow 1: Playwright Framework Setup

1. Installation
   bash

   npm init playwright@latest
   # Select: TypeScript, GitHub Actions, Install Browsers

2. Configuration (

   playwright.config.ts

   )
   typescript

   import { defineConfig, devices } from '@playwright/test';
   
   export default defineConfig({
     testDir: './tests',
     fullyParallel: true,
     forbidOnly: !!process.env.CI,
     retries: process.env.CI ? 2 : 0,
     workers: process.env.CI ? 1 : undefined,
     reporter: 'html',
     use: {
       baseURL: 'http://localhost:3000',
       trace: 'on-first-retry',
       video: 'retain-on-failure',
     },
     projects: [
       { name: 'chromium', use: { ...devices['Desktop Chrome'] } },
       { name: 'firefox', use: { ...devices['Desktop Firefox'] } },
       { name: 'webkit', use: { ...devices['Desktop Safari'] } },
     ],
   });

3. First Test (

   tests/login.spec.ts

   )
   typescript

   import { test, expect } from '@playwright/test';
   
   test('has title', async ({ page }) => {
     await page.goto('/');
     await expect(page).toHaveTitle(/My App/);
   });
   
   test('login flow', async ({ page }) => {
     await page.goto('/login');
     await page.getByLabel('Email').fill('user@example.com');
     await page.getByLabel('Password').fill('password');
     await page.getByRole('button', { name: 'Sign in' }).click();
     await expect(page).toHaveURL('/dashboard');
   });

Workflow 3: API Testing (Integration)

1. API Context
   typescript

   test('create user via API', async ({ request }) => {
     const response = await request.post('/api/users', {
       data: {
         name: 'Test User',
         email: `test-${Date.now()}@example.com`
       }
     });
     
     expect(response.ok()).toBeTruthy();
     const body = await response.json();
     expect(body.id).toBeDefined();
   });

Workflow 5: Playwright Sharding (CI Parallelism)

1. GitHub Actions Strategy
   yaml

   strategy:
     fail-fast: false
     matrix:
       shardIndex: [1, 2, 3, 4]
       shardTotal: [4]

2. Run Command
   bash

   npx playwright test --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}

3. Merge Reports
   • Upload

     blob-report

     artifact from each shard.
   • Use

     npx playwright merge-reports --reporter html ./all-blob-reports

     .

5. Anti-Patterns & Gotchas

❌ Anti-Pattern 1: Hardcoded Waits

• await page.waitForTimeout(5000);

• Slows down tests unnecessarily.
• Flaky if the app takes 5.1 seconds.

• Use Auto-waiting assertions:

  await expect(locator).toBeVisible();

  .
• Use

  await page.waitForURL()

  .

❌ Anti-Pattern 2: Test Interdependency

• Test B relies on Test A creating a user.

• If Test A fails, Test B fails (Cascade).
• Cannot run tests in parallel (Sharding impossible).

• Isolation: Each test must setup its own data (via API or Fixtures).

❌ Anti-Pattern 3: Selecting by CSS Classes

• page.locator('.btn-primary')

• Classes change for styling.
• Not accessible.

• User-facing locators:

  getByRole('button', { name: 'Submit' })

  ,

  getByLabel('Email')

  ,

  getByTestId('submit-btn')

  .

7. Quality Checklist

• Flakiness: No flaky tests allowed in

  main

  . Quarantined if flaky.
• Retries: configured for CI (max 2).
• Isolation: Tests run in parallel without data collision.

• POM: Page Object Model used for shared UI logic.
• Locators: User-facing locators used (Role, Text, Label).
• Base URL: Configured in config, not hardcoded.

• Critical Paths: Login, Checkout, core features covered.
• Browsers: Tested on Chromium, Firefox, WebKit.
• Mobile: Tested on Mobile Viewport (Emulation).

Anti-Patterns

Test Design Anti-Patterns

• Brittle Selectors: Using fragile CSS/XPath selectors - use semantic locators
• Test Data Coupling: Tests depending on specific test data - use data factories
• Long Test Chains: Tests doing too much - single responsibility per test
• Hardcoded Waits: Using sleep/fixed waits - use dynamic waiting strategies

Flaky Test Anti-Patterns

• Network Assumptions: Tests assuming network reliability - mock external services
• Timing Dependencies: Tests sensitive to execution timing - remove timing dependencies
• State Leakage: Tests affecting each other - ensure test isolation
• Race Conditions: Tests hitting race conditions - add proper synchronization

Maintenance Anti-Patterns

• Page Object Bloat: Page objects becoming too large - split and refactor
• Hardcoded URLs: URLs embedded in tests - use configuration
• No Page Transitions: Tests not handling navigation - proper flow handling
• Snapshot Drift: Unreviewed visual changes - always review visual diffs

CI/CD Anti-Patterns

• Long Test Runs: Tests taking too long - parallelize and optimize
• No Test Retry: Flaky tests failing builds - implement intelligent retries
• Environment Mismatch: Tests passing locally but failing in CI - containerize environments
• No Test Metrics: Not tracking test health - monitor flakiness and coverage