setup-sdk-testing

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

setup-sdk-testing

搭建SDK测试环境

Set up and run tests for Speakeasy-generated SDKs using contract testing, custom Arazzo workflows, or integration tests against live APIs.

使用契约测试、自定义Arazzo工作流或针对真实API的集成测试，为Speakeasy生成的SDK搭建并运行测试。

Content Guides

内容指南

Topic	Guide
Arazzo Reference	content/arazzo-reference.md

The Arazzo reference provides complete syntax for workflows, steps, success criteria, environment variables, and chaining operations.

主题	指南
Arazzo参考文档	content/arazzo-reference.md

Arazzo参考文档提供了工作流、步骤、成功条件、环境变量以及链式操作的完整语法说明。

When to Use

适用场景

Setting up automated testing for a generated SDK
Enabling contract test generation via
```
gen.yaml
```
Writing custom multi-step API workflow tests with Arazzo
Configuring integration tests against a live API
Debugging
```
ResponseValidationError
```
or test failures
User says: "test SDK", "contract testing", "Arazzo tests", "speakeasy test", "mock server", "test generation"

为生成的SDK设置自动化测试
通过
```
gen.yaml
```
启用契约测试生成
使用Arazzo编写自定义多步骤API工作流测试
配置针对真实API的集成测试
调试
```
ResponseValidationError
```
或测试失败问题
用户提及："test SDK"、"contract testing"、"Arazzo tests"、"speakeasy test"、"mock server"、"test generation"

Inputs

输入参数

Input	Required	Description
`gen.yaml`	Yes	Generation config; contract testing is enabled here
OpenAPI spec	Yes	The API spec the SDK is generated from
Target language	Yes	`typescript` , `python` , `go` , or `java` (contract test support)
`.speakeasy/tests.arazzo.yaml`	No	Custom Arazzo test definitions
Live API credentials	No	Required only for integration testing

输入项	是否必填	描述
`gen.yaml`	是	生成配置文件；契约测试在此处启用
OpenAPI规范	是	生成SDK所基于的API规范
目标语言	是	`typescript` 、 `python` 、 `go` 或 `java` （支持契约测试）
`.speakeasy/tests.arazzo.yaml`	否	自定义Arazzo测试定义文件
真实API凭据	否	仅集成测试时需要

Outputs

输出结果

Output	Location
Generated contract tests	`tests/` directory in SDK output
Mock server config	Auto-generated alongside contract tests
Arazzo test results	Terminal output from `speakeasy test`
CI workflow	`.github/workflows/` (if integration testing configured)

输出项	位置
生成的契约测试	SDK输出目录下的 `tests/` 文件夹
Mock服务器配置	随契约测试自动生成
Arazzo测试结果	`speakeasy test` 命令的终端输出
CI工作流	`.github/workflows/` （若配置了集成测试）

Prerequisites

前置条件

Speakeasy CLI installed and authenticated (
```
speakeasy auth login
```
)
A working SDK generation setup (
```
gen.yaml
```
+ OpenAPI spec)
For integration tests: API credentials as environment variables

已安装并认证Speakeasy CLI（执行
```
speakeasy auth login
```
）
已完成SDK生成的基础设置（
```
gen.yaml
```
+ OpenAPI规范）
集成测试：需将API凭据设置为环境变量

Decision Framework

决策框架

Pick the right testing approach based on what you need:

Need	Approach	Effort
Verify SDK types and methods match the API contract	Contract testing	Low (auto-generated)
Test multi-step API workflows (create then verify)	Custom Arazzo tests	Medium
Validate against a live API with real data	Integration testing	High
Catch regressions on every SDK regeneration	Contract testing + CI	Low
Test authentication flows end-to-end	Integration testing	High
Verify chained operations with data dependencies	Custom Arazzo tests	Medium

Start with contract testing. It is auto-generated and catches the most common issues. Add custom Arazzo tests for workflow coverage, and integration tests only when live API validation is required.

根据需求选择合适的测试方案：

需求	方案	工作量
验证SDK的类型和方法是否与API契约一致	契约测试	低（自动生成）
测试多步骤API工作流（创建后验证）	自定义Arazzo测试	中
使用真实数据针对真实API进行验证	集成测试	高
每次SDK重新生成时捕获回归问题	契约测试 + CI	低
端到端测试认证流程	集成测试	高
验证带有数据依赖的链式操作	自定义Arazzo测试	中

优先使用契约测试。它是自动生成的，能捕获大多数常见问题。针对工作流覆盖添加自定义Arazzo测试，仅当需要真实API验证时再使用集成测试。

Command

命令

Enable Contract Testing

启用契约测试

Add to

gen.yaml

yaml

generation:
  tests:
    generateTests: true

Then regenerate the SDK:

bash

speakeasy run --output console

在

gen.yaml

中添加以下配置：

yaml

generation:
  tests:
    generateTests: true

然后重新生成SDK：

bash

speakeasy run --output console

Run Tests

运行测试

bash

undefined

bash

undefined

Run all Arazzo-defined tests (contract + custom)

运行所有Arazzo定义的测试（契约测试 + 自定义测试）

speakeasy test

Run tests for a specific target

针对特定目标运行测试

speakeasy test --target my-typescript-sdk

Run with verbose output for debugging

启用详细输出以调试

speakeasy test --verbose

undefined

speakeasy test --verbose

undefined

Run via CI

通过CI运行测试

Contract tests run automatically in the Speakeasy GitHub Actions workflow when test generation is enabled. No additional CI configuration is needed for contract tests.

当启用测试生成后，契约测试会在Speakeasy GitHub Actions工作流中自动运行。契约测试无需额外的CI配置。

Example

示例

1. Contract Testing (Quick Start)

1. 契约测试（快速入门）

Enable test generation in

gen.yaml

yaml

generation:
  tests:
    generateTests: true

Regenerate the SDK:

bash

speakeasy run --output console

Run the generated tests:

bash

speakeasy test

The CLI generates tests from your OpenAPI spec, creates a mock server that returns spec-compliant responses, and validates that the SDK correctly handles requests and responses.

在

gen.yaml

中启用测试生成：

yaml

generation:
  tests:
    generateTests: true

重新生成SDK：

bash

speakeasy run --output console

运行生成的测试：

bash

speakeasy test

CLI会根据你的OpenAPI规范生成测试，创建一个返回符合规范响应的Mock服务器，并验证SDK是否能正确处理请求和响应。

2. Custom Arazzo Tests

2. 自定义Arazzo测试

Create or edit

.speakeasy/tests.arazzo.yaml

yaml

arazzo: 1.0.0
info:
  title: Custom SDK Tests
  version: 1.0.0

sourceDescriptions:
  - name: my-api
    type: openapi
    url: ./openapi.yaml

workflows:
  - workflowId: create-and-verify-resource
    steps:
      - stepId: create-resource
        operationId: createResource
        requestBody:
          payload:
            name: "test-resource"
            type: "example"
        successCriteria:
          - condition: $statusCode == 201
        outputs:
          resourceId: $response.body#/id

      - stepId: get-resource
        operationId: getResource
        parameters:
          - name: id
            in: path
            value: $steps.create-resource.outputs.resourceId
        successCriteria:
          - condition: $statusCode == 200
          - condition: $response.body#/name == "test-resource"

  - workflowId: list-and-filter
    steps:
      - stepId: list-resources
        operationId: listResources
        parameters:
          - name: limit
            in: query
            value: 10
        successCriteria:
          - condition: $statusCode == 200

Run the custom tests:

bash

speakeasy test

创建或编辑

.speakeasy/tests.arazzo.yaml

：

yaml

arazzo: 1.0.0
info:
  title: Custom SDK Tests
  version: 1.0.0

sourceDescriptions:
  - name: my-api
    type: openapi
    url: ./openapi.yaml

workflows:
  - workflowId: create-and-verify-resource
    steps:
      - stepId: create-resource
        operationId: createResource
        requestBody:
          payload:
            name: "test-resource"
            type: "example"
        successCriteria:
          - condition: $statusCode == 201
        outputs:
          resourceId: $response.body#/id

      - stepId: get-resource
        operationId: getResource
        parameters:
          - name: id
            in: path
            value: $steps.create-resource.outputs.resourceId
        successCriteria:
          - condition: $statusCode == 200
          - condition: $response.body#/name == "test-resource"

  - workflowId: list-and-filter
    steps:
      - stepId: list-resources
        operationId: listResources
        parameters:
          - name: limit
            in: query
            value: 10
        successCriteria:
          - condition: $statusCode == 200

运行自定义测试：

bash

speakeasy test

Using Environment Variables in Arazzo Tests

在Arazzo测试中使用环境变量

Reference environment variables for sensitive values:

yaml

steps:
  - stepId: authenticated-request
    operationId: getProtectedResource
    parameters:
      - name: Authorization
        in: header
        value: Bearer $env.API_TOKEN
    successCriteria:
      - condition: $statusCode == 200

引用环境变量存储敏感值：

yaml

steps:
  - stepId: authenticated-request
    operationId: getProtectedResource
    parameters:
      - name: Authorization
        in: header
        value: Bearer $env.API_TOKEN
    successCriteria:
      - condition: $statusCode == 200

Disabling a Specific Test

禁用特定测试

Use an overlay to disable a generated test without deleting it:

yaml

overlay: 1.0.0
info:
  title: Disable flaky test
actions:
  - target: $["workflows"][?(@.workflowId=="flaky-test")]
    update:
      x-speakeasy-test:
        disabled: true

使用覆盖配置禁用生成的测试，无需删除文件：

yaml

overlay: 1.0.0
info:
  title: Disable flaky test
actions:
  - target: $["workflows"][?(@.workflowId=="flaky-test")]
    update:
      x-speakeasy-test:
        disabled: true

3. Integration Testing

3. 集成测试

For live API testing, use the SDK factory pattern:

TypeScript example:

typescript

import { SDK } from "./src";

function createTestClient(): SDK {
  return new SDK({
    apiKey: process.env.TEST_API_KEY,
    serverURL: process.env.TEST_API_URL ?? "https://api.example.com",
  });
}

describe("Integration Tests", () => {
  const client = createTestClient();

  it("should list resources", async () => {
    const result = await client.resources.list({ limit: 5 });
    expect(result.statusCode).toBe(200);
    expect(result.data).toBeDefined();
  });

  it("should create and delete resource", async () => {
    // Create
    const created = await client.resources.create({ name: "integration-test" });
    expect(created.statusCode).toBe(201);

    // Cleanup
    const deleted = await client.resources.delete({ id: created.data.id });
    expect(deleted.statusCode).toBe(204);
  });
});

Python example:

python

import os
import pytest
from my_sdk import SDK

@pytest.fixture
def client():
    return SDK(
        api_key=os.environ["TEST_API_KEY"],
        server_url=os.environ.get("TEST_API_URL", "https://api.example.com"),
    )

def test_list_resources(client):
    result = client.resources.list(limit=5)
    assert result.status_code == 200
    assert result.data is not None

@pytest.mark.cleanup
def test_create_and_delete(client):
    created = client.resources.create(name="integration-test")
    assert created.status_code == 201
    try:
        fetched = client.resources.get(id=created.data.id)
        assert fetched.data.name == "integration-test"
    finally:
        client.resources.delete(id=created.data.id)

GitHub Actions CI for integration tests:

yaml

name: Integration Tests
on:
  schedule:
    - cron: "0 6 * * 1-5"
  workflow_dispatch:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: speakeasy-api/sdk-generation-action@v15
        with:
          speakeasy_version: latest
      - run: speakeasy test
        env:
          SPEAKEASY_API_KEY: ${{ secrets.SPEAKEASY_API_KEY }}
      - name: Run integration tests
        run: npm test -- --grep "integration"
        env:
          TEST_API_KEY: ${{ secrets.TEST_API_KEY }}
          TEST_API_URL: ${{ secrets.TEST_API_URL }}

针对真实API测试时，使用SDK工厂模式：

TypeScript示例：

typescript

import { SDK } from "./src";

function createTestClient(): SDK {
  return new SDK({
    apiKey: process.env.TEST_API_KEY,
    serverURL: process.env.TEST_API_URL ?? "https://api.example.com",
  });
}

describe("Integration Tests", () => {
  const client = createTestClient();

  it("should list resources", async () => {
    const result = await client.resources.list({ limit: 5 });
    expect(result.statusCode).toBe(200);
    expect(result.data).toBeDefined();
  });

  it("should create and delete resource", async () => {
    // Create
    const created = await client.resources.create({ name: "integration-test" });
    expect(created.statusCode).toBe(201);

    // Cleanup
    const deleted = await client.resources.delete({ id: created.data.id });
    expect(deleted.statusCode).toBe(204);
  });
});

Python示例：

python

import os
import pytest
from my_sdk import SDK

@pytest.fixture
def client():
    return SDK(
        api_key=os.environ["TEST_API_KEY"],
        server_url=os.environ.get("TEST_API_URL", "https://api.example.com"),
    )

def test_list_resources(client):
    result = client.resources.list(limit=5)
    assert result.status_code == 200
    assert result.data is not None

@pytest.mark.cleanup
def test_create_and_delete(client):
    created = client.resources.create(name="integration-test")
    assert created.status_code == 201
    try:
        fetched = client.resources.get(id=created.data.id)
        assert fetched.data.name == "integration-test"
    finally:
        client.resources.delete(id=created.data.id)

集成测试的GitHub Actions CI配置：

yaml

name: Integration Tests
on:
  schedule:
    - cron: "0 6 * * 1-5"
  workflow_dispatch:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: speakeasy-api/sdk-generation-action@v15
        with:
          speakeasy_version: latest
      - run: speakeasy test
        env:
          SPEAKEASY_API_KEY: ${{ secrets.SPEAKEASY_API_KEY }}
      - name: Run integration tests
        run: npm test -- --grep "integration"
        env:
          TEST_API_KEY: ${{ secrets.TEST_API_KEY }}
          TEST_API_URL: ${{ secrets.TEST_API_URL }}

What NOT to Do

注意事项

Do NOT skip contract testing and jump straight to integration tests -- contract tests are free and catch most issues
Do NOT hardcode API keys or secrets in Arazzo test files -- use
```
$env.VARIABLE_NAME
```
syntax
Do NOT modify auto-generated test files directly -- they are overwritten on regeneration; use custom Arazzo tests instead
Do NOT disable failing contract tests without investigating -- a
```
ResponseValidationError
```
usually means the spec and API are out of sync
Do NOT run integration tests with destructive operations in production -- always use a test/staging environment
Do NOT assume all languages support contract testing -- currently supported: TypeScript, Python, Go, Java

请勿跳过契约测试直接使用集成测试——契约测试无需额外成本，能捕获大多数问题
请勿在Arazzo测试文件中硬编码API密钥或敏感信息——使用
```
$env.VARIABLE_NAME
```
语法
请勿直接修改自动生成的测试文件——重新生成SDK时这些文件会被覆盖；请使用自定义Arazzo测试
请勿在未调查原因的情况下禁用失败的契约测试——
```
ResponseValidationError
```
通常意味着规范与API不一致
请勿在生产环境中运行带有破坏性操作的集成测试——始终使用测试/预发布环境
请勿假设所有语言都支持契约测试——当前支持：TypeScript、Python、Go、Java

Troubleshooting

故障排除

ResponseValidationError

in contract tests

契约测试中出现

ResponseValidationError

The SDK response does not match the OpenAPI spec. Common causes:

Spec is outdated: Regenerate from the latest API spec
Missing required fields: Check your spec's
```
required
```
arrays match actual API responses
Type mismatches: Verify
```
type
```
and
```
format
```
fields in schema definitions

bash

undefined

SDK响应与OpenAPI规范不匹配。常见原因：

规范已过时：使用最新的API规范重新生成SDK
缺少必填字段：检查规范中的
```
required
```
数组是否与实际API响应一致
类型不匹配：验证模式定义中的
```
type
```
和
```
format
```
字段

bash

undefined

Regenerate with latest spec and re-run tests

使用最新规范重新生成并重新运行测试

speakeasy run --output console && speakeasy test --verbose

undefined

speakeasy run --output console && speakeasy test --verbose

undefined

Tests pass locally but fail in CI

本地测试通过但CI中失败

Check that all environment variables are set in CI secrets
Verify the CI runner has network access to mock server ports
Ensure the Speakeasy CLI version matches between local and CI

检查CI环境中是否设置了所有环境变量
验证CI运行器是否能访问Mock服务器端口
确保本地和CI环境使用的Speakeasy CLI版本一致

speakeasy test

command not found

speakeasy test

命令未找到

Update the Speakeasy CLI:

bash

speakeasy update

更新Speakeasy CLI：

bash

speakeasy update

Mock server fails to start

Mock服务器启动失败

Check for port conflicts on the default mock server port
Ensure the OpenAPI spec is valid:
```
speakeasy lint openapi -s spec.yaml
```
Verify
```
generateTests: true
```
is set in
```
gen.yaml
```
and the SDK has been regenerated

检查默认Mock服务器端口是否存在端口冲突
确保OpenAPI规范有效：执行
```
speakeasy lint openapi -s spec.yaml
```
验证
```
gen.yaml
```
中是否设置了
```
generateTests: true
```
，且SDK已重新生成

Custom Arazzo test not running

自定义Arazzo测试未运行

Verify the file is at
```
.speakeasy/tests.arazzo.yaml
```
Check that
```
operationId
```
values match those in your OpenAPI spec exactly
Validate YAML syntax -- indentation errors are the most common cause

验证文件路径是否为
```
.speakeasy/tests.arazzo.yaml
```
检查
```
operationId
```
值是否与OpenAPI规范中的完全一致
验证YAML语法——缩进错误是最常见的原因

Integration tests intermittently fail

集成测试间歇性失败

Add retry logic for network-dependent tests
Use unique resource names with timestamps to avoid collisions
Ensure cleanup runs even on test failure (use
```
finally
```
or
```
afterEach
```
)

为依赖网络的测试添加重试逻辑
使用带时间戳的唯一资源名称避免冲突
确保即使测试失败也会执行清理操作（使用
```
finally
```
或
```
afterEach
```
）

setup-sdk-testing

Original

Translation

setup-sdk-testing

搭建SDK测试环境

Content Guides

内容指南

When to Use

适用场景

Inputs

输入参数

Outputs

输出结果

Prerequisites

前置条件

Decision Framework

决策框架

Command

命令

Enable Contract Testing

启用契约测试

Run Tests

运行测试

Run all Arazzo-defined tests (contract + custom)

运行所有Arazzo定义的测试（契约测试 + 自定义测试）

Run tests for a specific target

针对特定目标运行测试

Run with verbose output for debugging

启用详细输出以调试

Run via CI

通过CI运行测试

Example

示例

1. Contract Testing (Quick Start)

1. 契约测试（快速入门）

2. Custom Arazzo Tests

2. 自定义Arazzo测试

Using Environment Variables in Arazzo Tests

在Arazzo测试中使用环境变量

Disabling a Specific Test

禁用特定测试

3. Integration Testing

3. 集成测试

What NOT to Do

注意事项

Troubleshooting

故障排除

ResponseValidationError in contract tests

契约测试中出现ResponseValidationError

Regenerate with latest spec and re-run tests

使用最新规范重新生成并重新运行测试

Tests pass locally but fail in CI

本地测试通过但CI中失败

speakeasy test command not found

speakeasy test命令未找到

Mock server fails to start

Mock服务器启动失败

Custom Arazzo test not running

自定义Arazzo测试未运行

Integration tests intermittently fail

集成测试间歇性失败

`ResponseValidationError`
in contract tests

契约测试中出现
`ResponseValidationError`

`speakeasy test`
command not found

`speakeasy test`
命令未找到