run-tests

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Test Suite Validation Skill

测试套件验证技能

This skill helps you efficiently validate code changes by running the appropriate subset of the test suite. It uses

scripts/run-tests

to intelligently discover affected tests and run only what's necessary for validation.

该技能可帮助你通过运行测试套件的合适子集，高效验证代码变更。它使用

scripts/run-tests

智能发现受影响的测试，仅运行验证所需的必要测试。

When to Use This Skill

何时使用该技能

Use this skill when you have:

Made changes to source code files and want to validate they work
Fixed a bug and want to verify the fix
Added a feature and need test coverage
Modified test infrastructure or configuration
Want to verify changes don't break existing functionality

当你遇到以下情况时使用该技能：

修改了源代码文件，想要验证其是否正常工作
修复了bug，想要验证修复效果
添加了新功能，需要测试覆盖
修改了测试基础设施或配置
想要验证变更不会破坏现有功能

Key Principles

核心原则

Always use the run-tests skill when testing code changes - it's optimized for intelligent suite discovery
Never run pytest directly - bypasses the project's test infrastructure (use
```
scripts/run-tests
```
or
```
riot
```
via
```
scripts/ddtest
```
)
Minimal venvs for iteration - run 1-2 venvs initially, expand only if needed
Use
--dry-run
first - see what would run before executing
Follow official docs -
```
docs/contributing-testing.rst
```
is the source of truth for testing procedures

测试代码变更时始终使用run-tests技能——它针对智能套件发现进行了优化
切勿直接运行pytest——这会绕过项目的测试基础设施（请使用
```
scripts/run-tests
```
或通过
```
scripts/ddtest
```
调用
```
riot
```
）
迭代时使用最少的venv——最初运行1-2个venv，仅在需要时扩展
先使用
--dry-run
参数——在执行前查看会运行哪些内容
遵循官方文档——
```
docs/contributing-testing.rst
```
是测试流程的权威来源

How This Skill Works

该技能的工作原理

Step 1: Identify Changed Files

步骤1：识别已变更文件

First, determine which files were modified:

If you have pending edits, I'll identify the changed files from the current session
I'll look at git status to find staged, unstaged, and untracked changes
You can also specify files explicitly if working on specific changes

首先，确定哪些文件被修改：

如果你有未提交的编辑，我会从当前会话中识别已变更文件
我会查看git状态，找出已暂存、未暂存和未跟踪的变更
如果你在处理特定变更，也可以明确指定文件

Step 2: Discover Available Test Suites

步骤2：发现可用测试套件

I'll use the

scripts/run-tests

script to discover what test suites match your changes:

bash

scripts/run-tests --list <edited-files>

This outputs JSON showing:

Available test suites that match your changed files
All venvs (Python versions + package combinations) available for each suite
Their hashes, Python versions, and package versions

我会使用

scripts/run-tests

脚本发现与你的变更匹配的测试套件：

bash

scripts/run-tests --list <edited-files>

该命令会输出JSON格式的结果，包含：

与已变更文件匹配的可用测试套件
每个套件可用的所有venv（Python版本+软件包组合）
它们的哈希值、Python版本和软件包版本

Step 3: Intelligently Select Venvs

步骤3：智能选择venv

Rather than running ALL available venvs (which could take hours), I'll select the minimal set needed to validate your changes:

我不会运行所有可用的venv（这可能需要数小时），而是会选择最小集合来验证你的变更：

For Core/Tracing Changes (Broad Impact)

针对核心/追踪变更（影响范围广）

When you modify files like:

ddtrace/internal/core/*

ddtrace/_trace/*

ddtrace/trace/*

```
ddtrace/_monkey.py
```
,
```
ddtrace/settings/*
```
```
ddtrace/constants.py
```

Strategy: Run core tracer + internal tests with 1 venv each

Example:
```
tracer
```
suite with latest Python +
```
internal
```
suite with latest Python
This validates broad-reaching changes without excessive overhead
Skip integration suites unless the change directly affects integration code

当你修改以下文件时：

ddtrace/internal/core/*

ddtrace/_trace/*

ddtrace/trace/*

```
ddtrace/_monkey.py
```
,
```
ddtrace/settings/*
```
```
ddtrace/constants.py
```

策略： 为核心追踪器和内部测试各运行1个venv

示例：使用最新Python版本运行
```
tracer
```
套件 + 使用最新Python版本运行
```
internal
```
套件
这样无需过多开销即可验证影响范围广的变更
除非变更直接影响集成代码，否则跳过集成套件

For Integration/Contrib Changes (Targeted Impact)

针对集成/贡献代码变更（影响范围明确）

When you modify files like:

ddtrace/contrib/flask/*

ddtrace/contrib/django/*

, etc.

```
ddtrace/contrib/*/patch.py
```
or integration-specific code

Strategy: Run ONLY the affected integration suite with 1-2 venvs

Example: Flask changes → run
```
contrib::flask
```
suite with latest Python
If change involves multiple versions (e.g., Django 3.x and 4.x), pick 1 venv per major version
Skip unrelated integrations

当你修改以下文件时：

ddtrace/contrib/flask/*

ddtrace/contrib/django/*

等

```
ddtrace/contrib/*/patch.py
```
或集成特定代码

策略： 仅运行受影响的集成套件，使用1-2个venv

示例：Flask变更 → 使用最新Python版本运行
```
contrib::flask
```
套件
如果变更涉及多个版本（如Django 3.x和4.x），为每个主要版本选择1个venv
跳过无关的集成套件

For Test-Only Changes

针对仅测试文件的变更

When you modify

tests/

files (but not test infrastructure):

Run only the specific test files/functions modified
Use pytest args:
```
-- -k test_name
```
or direct test file paths

当你修改

tests/

目录下的文件（但不修改测试基础设施）：

仅运行被修改的特定测试文件/函数
使用pytest参数：
```
-- -k test_name
```
或直接指定测试文件路径

For Test Infrastructure Changes

针对测试基础设施的变更

When you modify:

tests/conftest.py

tests/suitespec.yml

scripts/run-tests

riotfile.py

Strategy: Run a quick smoke test suite

Example:
```
internal
```
suite with 1 venv as a sanity check
Or run small existing test suites to verify harness changes

当你修改以下文件时：

tests/conftest.py

tests/suitespec.yml

scripts/run-tests

riotfile.py

策略： 运行快速冒烟测试套件

示例：使用1个venv运行
```
internal
```
套件作为 sanity check
或运行小型现有测试套件来验证测试工具的变更

Step 4: Execute Selected Venvs

步骤4：执行选定的venv

I'll run the selected venvs using:

bash

scripts/run-tests --venv <hash1> --venv <hash2> ...

This will:

Start required Docker services (redis, postgres, etc.)
Run tests in the specified venvs sequentially
Stop services after completion
Show real-time output and status

我会使用以下命令运行选定的venv：

bash

scripts/run-tests --venv <hash1> --venv <hash2> ...

该命令会：

启动所需的Docker服务（如redis、postgres等）
按顺序在指定的venv中运行测试
完成后停止服务
显示实时输出和状态

Step 5: Handle Results

步骤5：处理结果

If tests pass: ✅ Your changes are validated!

If tests fail: 🔴 I'll:

Show you the failure details
Identify which venv failed
Ask clarifying questions to understand the issue
Offer to run specific failing tests with more verbosity
Help iterate on fixes and re-run

For re-running specific tests:

bash

scripts/run-tests --venv <hash> -- -vv -k test_name

如果测试通过： ✅ 你的变更已验证通过！

如果测试失败： 🔴 我会：

向你展示失败详情
识别哪个venv测试失败
询问澄清问题以理解问题所在
提供运行特定失败测试的选项，并增加详细程度
帮助你迭代修复并重新运行测试

重新运行特定测试的命令：

bash

scripts/run-tests --venv <hash> -- -vv -k test_name

When Tests Fail

测试失败时的处理流程

When you encounter test failures, follow this systematic approach:

Review the failure details carefully - Don't just skim the error, understand what's actually failing
Understand what's failing - Don't blindly re-run; analyze the root cause
Make code changes - Fix the underlying issue
Re-run with more verbosity if needed - Use
```
-vv
```
or
```
-vvv
```
for detailed output
Iterate until tests pass - Repeat the process with each fix

当遇到测试失败时，请遵循以下系统化步骤：

仔细查看失败详情 - 不要只浏览错误信息，要真正理解实际失败的内容
明确失败原因 - 不要盲目重新运行；分析根本原因
修改代码 - 修复潜在问题
必要时增加详细程度重新运行 - 使用
```
-vv
```
或
```
-vvv
```
获取详细输出
迭代直至测试通过 - 每次修复后重复该过程

Venv Selection Strategy in Detail

Venv选择策略详解

Understanding Venv Hashes

理解Venv哈希值

From

scripts/run-tests --list

, you'll see output like:

json

{
  "suites": [
    {
      "name": "tracer",
      "venvs": [
        {
          "hash": "abc123",
          "python_version": "3.8",
          "packages": "..."
        },
        {
          "hash": "def456",
          "python_version": "3.11",
          "packages": "..."
        }
      ]
    }
  ]
}

从

scripts/run-tests --list

命令的输出中，你会看到类似如下内容：

json

{
  "suites": [
    {
      "name": "tracer",
      "venvs": [
        {
          "hash": "abc123",
          "python_version": "3.8",
          "packages": "..."
        },
        {
          "hash": "def456",
          "python_version": "3.11",
          "packages": "..."
        }
      ]
    }
  ]
}

Selection Rules

选择规则

Latest Python version is your default choice
- Unless your change specifically targets an older Python version
- Example: if fixing Python 3.8 compatibility, also test 3.8
One venv per suite is usually enough for iteration
- Only run multiple venvs per suite if:
  - Change impacts multiple Python versions differently
  - Testing package compatibility variations (e.g., Django 3.x vs 4.x)
  - Initial validation passed and you want broader coverage
Minimize total venvs
- 1-2 venvs total for small targeted changes
- 3-4 venvs maximum for broader changes
- Never run 10+ venvs for initial validation (save that for CI)
Consider test runtime
- Each venv can take 5-30 minutes depending on suite
- With 2 venvs you're looking at 10-60 minutes for iteration
- With 5 venvs you're looking at 25-150 minutes
- Scale appropriately for your patience and deadline

默认选择最新Python版本
- 除非你的变更专门针对旧版本Python
- 示例：如果修复Python 3.8兼容性问题，同时测试3.8版本
每个套件通常只需1个venv用于迭代
- 仅在以下情况下为每个套件运行多个venv：
  - 变更对多个Python版本有不同影响
  - 测试软件包兼容性差异（如Django 3.x vs 4.x）
  - 初始验证已通过，需要更广泛的覆盖
最小化venv总数
- 小型针对性变更使用1-2个venv
- 范围更广的变更最多使用3-4个venv
- 初始验证切勿运行10个以上venv（这是CI的工作）
考虑测试运行时间
- 每个venv的运行时间取决于套件，可能需要5-30分钟
- 2个venv的迭代时间约为10-60分钟
- 5个venv的迭代时间约为25-150分钟
- 根据你的耐心和截止日期适当调整

Using

--venv

Directly

直接使用

--venv

参数

When you have a specific venv hash you want to run, you can use it directly without specifying file paths:

bash

scripts/run-tests --venv e06abee

The

--venv

flag automatically searches all available venvs across all suites, so it works regardless of what files you have locally changed. This is useful when:

You know exactly which venv you want to test
You have unrelated local changes that would otherwise limit suite matching
You want to quickly re-run a specific venv without file path arguments

当你有想要运行的特定venv哈希值时，可以直接使用该参数，无需指定文件路径：

bash

scripts/run-tests --venv e06abee

--venv

参数会自动搜索所有套件中的所有可用venv，因此无论你本地有哪些文件变更，它都能正常工作。在以下情况下非常有用：

你确切知道要测试哪个venv
你有不相关的本地变更，否则会限制套件匹配
你想快速重新运行特定venv，无需指定文件路径参数

Examples

示例

Example 1: Fixing a Flask Integration Bug

示例1：修复Flask集成Bug

Changed file:

ddtrace/contrib/internal/flask/patch.py

bash

scripts/run-tests --list ddtrace/contrib/internal/flask/patch.py

已变更文件：

ddtrace/contrib/internal/flask/patch.py

bash

scripts/run-tests --list ddtrace/contrib/internal/flask/patch.py

Output shows: contrib::flask suite available

输出显示：contrib::flask套件可用

Select output (latest Python):

选择输出（最新Python版本）：

Suite: contrib::flask

套件：contrib::flask

Venv: hash=e06abee, Python 3.13, flask

Venv：hash=e06abee, Python 3.13, flask

Run with --venv directly (searches all venvs automatically)

直接使用--venv参数运行（自动搜索所有venv）

scripts/run-tests --venv e06abee

Runs just Flask integration tests

仅运行Flask集成测试

undefined

undefined

Example 2: Fixing a Core Tracing Issue

示例2：修复核心追踪问题

Changed file:

ddtrace/_trace/tracer.py

bash

scripts/run-tests --list ddtrace/_trace/tracer.py

已变更文件：

ddtrace/_trace/tracer.py

bash

scripts/run-tests --list ddtrace/_trace/tracer.py

Output shows: tracer suite, internal suite available

输出显示：tracer套件、internal套件可用

Select strategy:

选择策略：

- tracer: latest Python (e.g., abc123)

- tracer：最新Python版本（如abc123）

- internal: latest Python (e.g., def456)

- internal：最新Python版本（如def456）

Run with --venv directly (searches all venvs automatically)

直接使用--venv参数运行（自动搜索所有venv）

scripts/run-tests --venv abc123 --venv def456

Validates core tracer and internal components

验证核心追踪器和内部组件

undefined

undefined

Example 3: Fixing a Test-Specific Bug

示例3：修复测试特定Bug

Changed file:

tests/contrib/flask/test_views.py

bash

scripts/run-tests --list tests/contrib/flask/test_views.py

已变更文件：

tests/contrib/flask/test_views.py

bash

scripts/run-tests --list tests/contrib/flask/test_views.py

Output shows: contrib::flask suite

输出显示：contrib::flask套件

Run just the specific test:

仅运行特定测试：

scripts/run-tests --venv flask_py311 -- -vv tests/contrib/flask/test_views.py

undefined

scripts/run-tests --venv flask_py311 -- -vv tests/contrib/flask/test_views.py

undefined

Example 4: Iterating on a Failing Test

示例4：迭代修复失败的测试

First run shows one test failing:

bash

scripts/run-tests --venv flask_py311 -- -vv -k test_view_called_twice

首次运行显示某个测试失败：

bash

scripts/run-tests --venv flask_py311 -- -vv -k test_view_called_twice

Focused on the specific failing test with verbose output

聚焦于特定失败测试，并显示详细输出

undefined

undefined

Best Practices

最佳实践

DO ✅

建议✅

Start small: Run 1 venv first, expand only if needed
Be specific: Use pytest
```
-k
```
filter when re-running failures
Check git: Verify you're testing the right files with
```
git status
```
Read errors: Take time to understand test failures before re-running
Ask for help: When unclear what tests to run, ask me to analyze the changes

从小规模开始：先运行1个venv，仅在需要时扩展
明确目标：重新运行失败测试时使用pytest
```
-k
```
过滤器
检查git状态：使用
```
git status
```
验证你测试的是正确的文件
阅读错误信息：重新运行前花时间理解测试失败原因
寻求帮助：当不确定要运行哪些测试时，让我分析你的变更

DON'T ❌

不建议❌

Run all venvs initially: That's what CI is for
Skip the minimal set guidance: It's designed to save you time
Ignore service requirements: Some suites need Docker services up
Run tests without changes saved: Make sure edits are saved first
Iterate blindly: Understand what's failing before re-running

初始运行所有venv：这是CI的工作
忽略最小集合指导：该指导旨在为你节省时间
忽略服务要求：某些套件需要Docker服务运行
未保存变更就运行测试：确保先保存编辑内容
盲目迭代：重新运行前理解失败原因

Additional Testing Resources

额外测试资源

For comprehensive testing guidance, refer to the contributing documentation:

docs/contributing-testing.rst - Detailed testing guidelines
- What kind of tests to write (unit tests, integration tests, e2e tests)
- When to write tests (feature development, bug fixes)
- Where to put tests in the repository
- Prerequisites (Docker, uv)
- Complete
```
scripts/run-tests
```
  usage examples
- Riot environment management details
- Running specific test files and functions
- Test debugging strategies
docs/contributing.rst - PR and testing requirements
- All changes need tests or documented testing strategy
- How tests fit into the PR review process
- Testing expectations for different types of changes
docs/contributing-design.rst - Test architecture context
- How products, integrations, and core interact
- Where different types of tests should live
- Testing patterns for each library component

When to reference these docs:

First time writing tests for this project → Read
```
contributing-testing.rst
```
Understanding test requirements for PRs → Read
```
contributing.rst
```
Need context on test architecture → Read
```
contributing-design.rst
```

如需全面的测试指导，请参考贡献文档：

docs/contributing-testing.rst - 详细测试指南
- 应编写哪种类型的测试（单元测试、集成测试、端到端测试）
- 何时编写测试（功能开发、Bug修复）
- 测试文件在仓库中的存放位置
- 前置条件（Docker、uv）
- ```
scripts/run-tests
```
  完整使用示例
- Riot环境管理详情
- 运行特定测试文件和函数的方法
- 测试调试策略
docs/contributing.rst - PR和测试要求
- 所有变更都需要测试或有文档记录的测试策略
- 测试在PR审查流程中的作用
- 不同类型变更的测试预期
docs/contributing-design.rst - 测试架构背景
- 产品、集成和核心组件如何交互
- 不同类型测试的存放位置
- 每个库组件的测试模式

何时参考这些文档：

首次为该项目编写测试 → 阅读
```
contributing-testing.rst
```
理解PR的测试要求 → 阅读
```
contributing.rst
```
需要测试架构背景 → 阅读
```
contributing-design.rst
```

Troubleshooting

故障排除

Docker services won't start

Docker服务无法启动

bash

undefined

bash

undefined

Manually check/stop services:

手动检查/停止服务：

docker compose ps docker compose down

undefined

docker compose ps docker compose down

undefined

Can't find matching suites

找不到匹配的套件

Verify the file path is correct
Check
```
tests/suitespec.yml
```
to understand suite patterns
Your file might not be covered by any suite pattern yet

验证文件路径是否正确
查看
```
tests/suitespec.yml
```
以理解套件模式
你的文件可能尚未被任何套件模式覆盖

Test takes too long

测试运行时间过长

You may have selected too many venvs
Try running with just 1 venv
Use pytest
```
-k
```
to run subset of tests

你可能选择了过多venv
尝试仅运行1个venv
使用pytest
```
-k
```
参数运行测试子集

Technical Details

技术细节

Architecture

架构

The

scripts/run-tests

system:

Maps source files to test suites using patterns in
```
tests/suitespec.yml
```
Uses
```
riot
```
to manage multiple Python/package combinations as venvs
Each venv is a self-contained environment
Docker services are managed per suite lifecycle
Tests can pass optional pytest arguments with
```
--
```

scripts/run-tests

系统：

使用
```
tests/suitespec.yml
```
中的模式将源文件映射到测试套件
使用
```
riot
```
管理多个Python/软件包组合作为venv
每个venv都是独立的环境
Docker服务按套件生命周期管理
测试可通过
```
--
```
参数传递可选的pytest参数

Supported Suite Types

支持的套件类型

Primary suites for validation:

```
tracer
```
: Core tracing functionality tests
```
internal
```
: Internal component tests
```
contrib::*
```
: Integration with specific libraries (flask, django, etc.)
```
integration_*
```
: Cross-library integration scenarios
Specialized:
```
telemetry
```
,
```
profiling
```
,
```
appsec
```
,
```
llmobs
```
, etc.

用于验证的主要套件：

```
tracer
```
：核心追踪功能测试
```
internal
```
：内部组件测试
```
contrib::*
```
：与特定库的集成测试（flask、django等）
```
integration_*
```
：跨库集成场景测试
专用套件：
```
telemetry
```
、
```
profiling
```
、
```
appsec
```
、
```
llmobs
```
等

Environment Variables

环境变量

Some suites require environment setup:

```
DD_TRACE_AGENT_URL
```
: For snapshot-based tests
Service-specific variables for Docker containers
These are handled automatically by the script

某些套件需要设置环境变量：

```
DD_TRACE_AGENT_URL
```
：用于基于快照的测试
Docker容器的服务特定变量
这些变量会由脚本自动处理