grey-haven-testing-strategy

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Grey Haven Testing Strategy

Grey Haven 测试策略

Comprehensive testing approach for TypeScript (Vitest) and Python (pytest) projects.

Follow these standards when writing tests, setting up test infrastructure, or improving test coverage in Grey Haven codebases.

适用于 TypeScript（Vitest）和 Python（pytest）项目的全面测试方法。

在 Grey Haven 代码库中编写测试、搭建测试基础设施或提升测试覆盖率时，请遵循以下标准。

Supporting Documentation

Testing Philosophy

测试理念

Coverage Requirements

覆盖率要求

Minimum: 80% code coverage for all projects (enforced in CI)
Target: 90%+ coverage for critical paths
100% coverage for security-critical code (auth, payments, multi-tenant isolation)

最低要求：80% 代码覆盖率（CI 中强制执行）
目标：关键路径 90%+ 覆盖率
安全关键代码 100% 覆盖率（认证、支付、多租户隔离模块）

Test Types (Markers)

测试类型（标记）

Grey Haven uses consistent test markers across languages:

unit: Fast, isolated tests of single functions/classes
integration: Tests involving multiple components or external dependencies
e2e: End-to-end tests through full user flows
benchmark: Performance tests measuring speed/memory

Grey Haven 在各语言中使用统一的测试标记：

unit：快速、独立的单一函数/类测试
integration：涉及多个组件或外部依赖的测试
e2e：覆盖完整用户流程的端到端测试
benchmark：测量速度/内存的性能测试

TypeScript Testing (Vitest)

TypeScript 测试（Vitest）

Quick Setup

快速搭建

Project Structure:

tests/
├── unit/                    # Fast, isolated tests
├── integration/             # Multi-component tests
└── e2e/                    # Playwright tests

Key Configuration:

typescript

// vitest.config.ts
export default defineConfig({
  test: {
    globals: true,
    environment: "jsdom",
    setupFiles: ["./tests/setup.ts"],
    coverage: {
      thresholds: { lines: 80, functions: 80, branches: 80, statements: 80 },
    },
  },
});

Running Tests:

bash

bun run test                 # Run all tests
bun run test:coverage        # With coverage report
bun run test:watch           # Watch mode
bun run test:ui              # UI mode
bun run test tests/unit/     # Unit tests only

See EXAMPLES.md for complete test examples.

项目结构：

tests/
├── unit/                    # 快速、独立的测试
├── integration/             # 多组件测试
└── e2e/                    # Playwright 测试

核心配置：

typescript

// vitest.config.ts
export default defineConfig({
  test: {
    globals: true,
    environment: "jsdom",
    setupFiles: ["./tests/setup.ts"],
    coverage: {
      thresholds: { lines: 80, functions: 80, branches: 80, statements: 80 },
    },
  },
});

运行测试：

bash

bun run test                 # 运行所有测试
bun run test:coverage        # 生成覆盖率报告
bun run test:watch           # 监听模式
bun run test:ui              # UI 模式
bun run test tests/unit/     # 仅运行单元测试

完整测试示例请查看 EXAMPLES.md。

Python Testing (pytest)

Python 测试（pytest）

Quick Setup

快速搭建

Project Structure:

tests/
├── conftest.py             # Shared fixtures
├── unit/                   # @pytest.mark.unit
├── integration/            # @pytest.mark.integration
├── e2e/                   # @pytest.mark.e2e
└── benchmark/             # @pytest.mark.benchmark

Key Configuration:

toml

undefined

项目结构：

tests/
├── conftest.py             # 共享夹具
├── unit/                   # @pytest.mark.unit
├── integration/            # @pytest.mark.integration
├── e2e/                   # @pytest.mark.e2e
└── benchmark/             # @pytest.mark.benchmark

核心配置：

toml

undefined

pyproject.toml

[tool.pytest.ini_options] addopts = ["--cov=app", "--cov-fail-under=80"] markers = [ "unit: Fast, isolated unit tests", "integration: Tests involving multiple components", "e2e: End-to-end tests through full flows", "benchmark: Performance tests", ]


**Running Tests:**

```bash


**运行测试：**

```bash

⚠️ ALWAYS activate virtual environment first!

⚠️ 请务必先激活虚拟环境！

source .venv/bin/activate

Run with Doppler for environment variables

使用 Doppler 管理环境变量运行测试

doppler run -- pytest # All tests doppler run -- pytest --cov=app # With coverage doppler run -- pytest -m unit # Unit tests only doppler run -- pytest -m integration # Integration tests only doppler run -- pytest -m e2e # E2E tests only doppler run -- pytest -v # Verbose output


**See [EXAMPLES.md](EXAMPLES.md#pytest-examples) for complete test examples.**

doppler run -- pytest # 所有测试 doppler run -- pytest --cov=app # 生成覆盖率报告 doppler run -- pytest -m unit # 仅运行单元测试 doppler run -- pytest -m integration # 仅运行集成测试 doppler run -- pytest -m e2e # 仅运行端到端测试 doppler run -- pytest -v # 详细输出


**完整测试示例请查看 [EXAMPLES.md](EXAMPLES.md#pytest-examples)。**

Test Markers Explained

测试标记说明

Unit Tests

单元测试

Characteristics:

Fast execution (< 100ms per test)
No external dependencies (database, API, file system)
Mock all external services
Test single function/class in isolation

Use for:

Utility functions
Business logic
Data transformations
Component rendering (React Testing Library)

特点：

执行速度快（每个测试 < 100ms）
无外部依赖（数据库、API、文件系统）
模拟所有外部服务
独立测试单一函数/类

适用场景：

工具函数
业务逻辑
数据转换
组件渲染（React Testing Library）

Integration Tests

集成测试

Characteristics:

Test multiple components together
May use real database/Redis (with cleanup)
Test API endpoints with FastAPI TestClient
Test React Query + server functions

Use for:

API endpoint flows
Database operations with repositories
Authentication flows
Multi-component interactions

特点：

测试多个组件协同工作
可使用真实数据库/Redis（需清理数据）
使用 FastAPI TestClient 测试 API 接口
测试 React Query + 服务器函数

适用场景：

API 接口流程
数据库操作与仓库层
认证流程
多组件交互

E2E Tests

端到端测试

Characteristics:

Test complete user flows
Use Playwright (TypeScript) or httpx (Python)
Test from user perspective
Slower execution (seconds per test)

Use for:

Registration/login flows
Critical user journeys
Form submissions
Multi-page workflows

特点：

测试完整用户流程
使用 Playwright（TypeScript）或 httpx（Python）
从用户视角测试
执行速度较慢（每个测试需数秒）

适用场景：

注册/登录流程
关键用户旅程
表单提交
多页面工作流

Benchmark Tests

基准测试

Characteristics:

Measure performance metrics
Track execution time
Monitor memory usage
Detect performance regressions

Use for:

Database query performance
Algorithm optimization
API response times
Batch operations

特点：

测量性能指标
跟踪执行时间
监控内存使用
检测性能退化

适用场景：

数据库查询性能
算法优化
API 响应时间
批量操作

Environment Variables with Doppler

使用 Doppler 管理环境变量

⚠️ CRITICAL: Grey Haven uses Doppler for ALL environment variables.

bash

undefined

⚠️ 重要提示：Grey Haven 所有环境变量均使用 Doppler 管理。

bash

undefined

Install Doppler

安装 Doppler

brew install dopplerhq/cli/doppler

Authenticate and setup

认证并配置

doppler login doppler setup

Run tests with Doppler

使用 Doppler 运行测试

doppler run -- bun run test # TypeScript doppler run -- pytest # Python

doppler run -- bun run test # TypeScript 测试 doppler run -- pytest # Python 测试

Use specific config

使用指定配置

doppler run --config test -- pytest


**Doppler provides:**
- `DATABASE_URL_TEST` - Test database connection
- `REDIS_URL` - Redis for tests (separate DB)
- `BETTER_AUTH_SECRET` - Auth secrets
- `STRIPE_SECRET_KEY` - External service keys (test mode)
- `PLAYWRIGHT_BASE_URL` - E2E test URL

**See [REFERENCE.md](REFERENCE.md#doppler-configuration) for complete setup.**

doppler run --config test -- pytest


**Doppler 提供以下变量：**
- `DATABASE_URL_TEST` - 测试数据库连接
- `REDIS_URL` - 测试用 Redis（独立数据库）
- `BETTER_AUTH_SECRET` - 认证密钥
- `STRIPE_SECRET_KEY` - 外部服务密钥（测试模式）
- `PLAYWRIGHT_BASE_URL` - 端到端测试地址

**完整配置请查看 [REFERENCE.md](REFERENCE.md#doppler-configuration)。**

Test Fixtures and Factories

测试夹具与工厂

TypeScript Factories

TypeScript 工厂

typescript

// tests/factories/user.factory.ts
import { faker } from "@faker-js/faker";

export function createMockUser(overrides = {}) {
  return {
    id: faker.string.uuid(),
    tenant_id: faker.string.uuid(),
    email_address: faker.internet.email(),
    name: faker.person.fullName(),
    ...overrides,
  };
}

typescript

// tests/factories/user.factory.ts
import { faker } from "@faker-js/faker";

export function createMockUser(overrides = {}) {
  return {
    id: faker.string.uuid(),
    tenant_id: faker.string.uuid(),
    email_address: faker.internet.email(),
    name: faker.person.fullName(),
    ...overrides,
  };
}

Python Fixtures

Python 夹具

python

undefined

python

undefined

tests/conftest.py

@pytest.fixture async def test_user(session, tenant_id): """Create test user with tenant isolation.""" user = User( tenant_id=tenant_id, email_address="test@example.com", name="Test User", ) session.add(user) await session.commit() return user


**See [EXAMPLES.md](EXAMPLES.md#test-factories-and-fixtures) for more patterns.**


**更多模式请查看 [EXAMPLES.md](EXAMPLES.md#test-factories-and-fixtures)。**

Multi-Tenant Testing

多租户测试

⚠️ ALWAYS test tenant isolation in multi-tenant projects:

python

@pytest.mark.unit
async def test_tenant_isolation(session, test_user, tenant_id):
    """Verify queries filter by tenant_id."""
    repo = UserRepository(session)

    # Should find with correct tenant
    user = await repo.get_by_id(test_user.id, tenant_id)
    assert user is not None

    # Should NOT find with different tenant
    different_tenant = uuid4()
    user = await repo.get_by_id(test_user.id, different_tenant)
    assert user is None

⚠️ 多租户项目中务必测试租户隔离：

python

@pytest.mark.unit
async def test_tenant_isolation(session, test_user, tenant_id):
    """Verify queries filter by tenant_id."""
    repo = UserRepository(session)

    # 使用正确租户 ID 应能找到用户
    user = await repo.get_by_id(test_user.id, tenant_id)
    assert user is not None

    # 使用错误租户 ID 不应找到用户
    different_tenant = uuid4()
    user = await repo.get_by_id(test_user.id, different_tenant)
    assert user is None

Continuous Integration

持续集成

GitHub Actions with Doppler:

yaml

undefined

结合 Doppler 的 GitHub Actions：

yaml

undefined

.github/workflows/test.yml

name: Run tests with Doppler env: DOPPLER_TOKEN: ${{ secrets.DOPPLER_TOKEN_TEST }} run: doppler run --config test -- bun run test:coverage


**See [REFERENCE.md](REFERENCE.md#github-actions-configuration) for complete workflow.**

name: Run tests with Doppler env: DOPPLER_TOKEN: ${{ secrets.DOPPLER_TOKEN_TEST }} run: doppler run --config test -- bun run test:coverage


**完整工作流请查看 [REFERENCE.md](REFERENCE.md#github-actions-configuration)。**

When to Apply This Skill

适用场景

Use this skill when:

✅ Writing new tests for features
✅ Setting up test infrastructure (Vitest/pytest)
✅ Configuring CI/CD test pipelines
✅ Debugging failing tests
✅ Improving test coverage (<80%)
✅ Reviewing test code quality
✅ Setting up Doppler for test environments
✅ Creating test fixtures and factories
✅ Implementing TDD workflow
✅ User mentions: "test", "testing", "pytest", "vitest", "coverage", "TDD", "unit test", "integration test", "e2e", "test setup", "CI testing"

在以下场景中使用本策略：

✅ 为新功能编写测试
✅ 搭建测试基础设施（Vitest/pytest）
✅ 配置 CI/CD 测试流水线
✅ 调试失败的测试
✅ 提升测试覆盖率（低于80%时）
✅ 评审测试代码质量
✅ 为测试环境配置 Doppler
✅ 创建测试夹具与工厂
✅ 实现 TDD 工作流
✅ 用户提及："test"、"testing"、"pytest"、"vitest"、"coverage"、"TDD"、"unit test"、"integration test"、"e2e"、"test setup"、"CI testing"

Template References

模板参考

These testing patterns come from Grey Haven production templates:

Frontend:
```
cvi-template
```
(Vitest + Playwright + React Testing Library)
Backend:
```
cvi-backend-template
```
(pytest + FastAPI TestClient + async fixtures)

这些测试模式源自 Grey Haven 生产环境模板：

前端：
```
cvi-template
```
（Vitest + Playwright + React Testing Library）
后端：
```
cvi-backend-template
```
（pytest + FastAPI TestClient + 异步夹具）

Critical Reminders

重要提醒

Coverage: 80% minimum (enforced in CI, blocks merge)
Test markers: unit, integration, e2e, benchmark (use consistently)
Doppler: ALWAYS use for test environment variables (never commit .env!)
Virtual env: MUST activate for Python tests (
```
source .venv/bin/activate
```
)
Tenant isolation: ALWAYS test multi-tenant scenarios
Fixtures: Use factories for test data generation (faker library)
Mocking: Mock external services in unit tests (use vi.mock or pytest mocks)
CI: Run tests with
```
doppler run --config test
```
Database: Use separate test database (Doppler provides
```
DATABASE_URL_TEST
```
)
Cleanup: Clean up test data after each test (use fixtures with cleanup)

覆盖率：最低 80%（CI 中强制执行，不达标将阻止合并）
测试标记：统一使用 unit、integration、e2e、benchmark
Doppler：测试环境变量务必使用 Doppler 管理（绝不要提交 .env 文件！）
虚拟环境：Python 测试必须激活虚拟环境（
```
source .venv/bin/activate
```
）
租户隔离：务必测试多租户场景的隔离性
夹具：使用工厂生成测试数据（faker 库）
模拟：单元测试中模拟外部服务（使用 vi.mock 或 pytest mocks）
CI：使用
```
doppler run --config test
```
运行测试
数据库：使用独立的测试数据库（Doppler 提供
```
DATABASE_URL_TEST
```
）
清理：每个测试后清理测试数据（使用带清理逻辑的夹具）

Next Steps

下一步操作

Need test examples? See EXAMPLES.md for copy-paste code
Need configurations? See REFERENCE.md for complete configs
Need templates? See templates/ for starter files
Need checklists? Use checklists/ for systematic test reviews
Need to run tests? Use scripts/ for helper utilities

需要测试示例？ 查看 EXAMPLES.md 获取可复制代码
需要配置文件？ 查看 REFERENCE.md 获取完整配置
需要模板？ 查看 templates/ 获取起始文件
需要检查清单？ 使用 checklists/ 进行系统化测试评审
需要运行测试？ 使用 scripts/ 中的辅助工具

grey-haven-testing-strategy

Original

Translation

Grey Haven Testing Strategy

Grey Haven 测试策略

Supporting Documentation

相关文档

Testing Philosophy

测试理念

Coverage Requirements

覆盖率要求

Test Types (Markers)

测试类型（标记）

TypeScript Testing (Vitest)

TypeScript 测试（Vitest）

Quick Setup

快速搭建

Python Testing (pytest)

Python 测试（pytest）

Quick Setup

快速搭建

pyproject.toml

pyproject.toml

⚠️ ALWAYS activate virtual environment first!

⚠️ 请务必先激活虚拟环境！

Run with Doppler for environment variables

使用 Doppler 管理环境变量运行测试

Test Markers Explained

测试标记说明

Unit Tests

单元测试

Integration Tests

集成测试

E2E Tests

端到端测试

Benchmark Tests

基准测试

Environment Variables with Doppler

使用 Doppler 管理环境变量

Install Doppler

安装 Doppler

Authenticate and setup

认证并配置

Run tests with Doppler

使用 Doppler 运行测试

Use specific config

使用指定配置

Test Fixtures and Factories

测试夹具与工厂

TypeScript Factories

TypeScript 工厂

Python Fixtures

Python 夹具

tests/conftest.py

tests/conftest.py

Multi-Tenant Testing

多租户测试

Continuous Integration

持续集成

.github/workflows/test.yml

.github/workflows/test.yml

When to Apply This Skill

适用场景

Template References

模板参考

Critical Reminders

重要提醒

Next Steps

下一步操作