launchdarkly-experiment-setup

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

LaunchDarkly Experiment Setup

LaunchDarkly 实验设置

You're using a skill that will guide you through setting up and running experiments in LaunchDarkly. Your job is to design the experiment, create it with the right metrics and treatments, start data collection, and verify it's running.

你正在使用一项技能，它将引导你在LaunchDarkly中设置并运行实验。你的任务是设计实验，创建包含正确指标和变体的实验，启动数据收集，并验证实验正在运行。

Prerequisites

前提条件

This skill requires the remotely hosted LaunchDarkly MCP server to be configured in your environment.

Required MCP tools:

```
create-experiment
```
-- create a new experiment with metrics and treatments
```
start-experiment-iteration
```
-- begin collecting data for the experiment
```
get-experiment
```
-- check experiment status and configuration

Optional MCP tools:

```
list-experiments
```
-- browse existing experiments in the project
```
update-experiment
```
-- modify experiment name or description
```
create-metric
```
-- create metrics if they don't exist yet
```
list-metrics
```
-- browse available metrics

本技能要求你的环境中已配置远程托管的LaunchDarkly MCP服务器。

必需的MCP工具：

```
create-experiment
```
-- 创建包含指标和变体的新实验
```
start-experiment-iteration
```
-- 开始为实验收集数据
```
get-experiment
```
-- 检查实验状态和配置

可选的MCP工具：

```
list-experiments
```
-- 浏览项目中的现有实验
```
update-experiment
```
-- 修改实验名称或描述
```
create-metric
```
-- 若指标不存在则创建新指标
```
list-metrics
```
-- 浏览可用指标

Core Concepts

核心概念

What Are Experiments?

什么是实验？

Experiments in LaunchDarkly let you measure the impact of feature flag variations on key metrics. An experiment consists of:

Treatments: The flag variations being compared (control vs. test)
Metrics: What you're measuring (conversion rate, latency, revenue, etc.)
Iterations: Data collection periods — start an iteration to begin collecting data
Holdout (optional): A percentage of traffic excluded from the experiment for baseline measurement

LaunchDarkly中的实验可让你衡量功能标志（feature flag）变体对关键指标的影响。一个实验包含：

变体（Treatments）：被比较的标志变体（对照组 vs 测试组）
指标（Metrics）：你要衡量的内容（转化率、延迟、收入等）
迭代（Iterations）：数据收集周期——启动迭代以开始收集数据
保留组（Holdout，可选）：排除在实验之外的流量百分比，用于基线测量

Experiment Lifecycle

实验生命周期

Create the experiment with metrics and treatments
Start an iteration to begin data collection
Monitor results as data accumulates
Stop the iteration when you have statistical significance
Ship the winning variation

创建包含指标和变体的实验
启动迭代以开始数据收集
监控数据积累过程中的结果
当获得统计显著性时停止迭代
发布获胜变体

Core Principles

核心原则

Metrics First: Ensure your metrics exist before creating the experiment
Clear Hypothesis: Know what you expect to improve and by how much
Proper Controls: Always include a control treatment (the current behavior)
Sufficient Sample Size: Let experiments run long enough for statistical significance
One Change at a Time: Test one variable per experiment for clear attribution

指标优先：创建实验前确保所需指标已存在
明确假设：清楚你期望改进什么以及改进幅度
适当对照：始终包含对照变体（当前行为）
足够样本量：让实验运行足够长的时间以获得统计显著性
一次一变量：每个实验仅测试一个变量，以便明确归因

Workflow

工作流程

Step 1: Prepare Metrics

步骤1：准备指标

Before creating an experiment, ensure the metrics you want to measure exist:

Use
```
list-metrics
```
to check for existing metrics
If needed, use
```
create-metric
```
to create new ones
Note the metric keys — you'll need them for the experiment

Common metric types:

Goal	Metric Type	Example
Conversion	Custom conversion	`checkout-completed`
Performance	Custom numeric	`page-load-time-ms`
Engagement	Custom conversion	`feature-clicked`
Revenue	Custom numeric	`order-value`

创建实验前，确保你要衡量的指标已存在：

使用
```
list-metrics
```
检查现有指标
如有需要，使用
```
create-metric
```
创建新指标
记录指标键——创建实验时需要用到它们

常见指标类型：

目标	指标类型	示例
转化	自定义转化	`checkout-completed`
性能	自定义数值	`page-load-time-ms`
参与度	自定义转化	`feature-clicked`
收入	自定义数值	`order-value`

Step 2: Create the Experiment

步骤2：创建实验

Use

create-experiment

with:

```
projectKey
```
and
```
environmentKey
```
-- where to run the experiment
```
name
```
-- descriptive name for the experiment
```
flagKey
```
-- the feature flag being experimented on
```
metrics
```
-- array of metric objects with
```
key
```
and
```
isGroup
```
fields
```
treatments
```
-- array of treatments, each with a
```
name
```
,
```
baseline
```
flag, and
```
parameters
```
```
holdout
```
(optional) -- percentage of traffic to exclude

json

{
  "projectKey": "my-project",
  "environmentKey": "production",
  "name": "Checkout Flow v2 Experiment",
  "flagKey": "checkout-flow-v2",
  "metrics": [
    {"key": "checkout-completed", "isGroup": false},
    {"key": "checkout-time-seconds", "isGroup": false}
  ],
  "treatments": [
    {
      "name": "Control",
      "baseline": true,
      "parameters": {
        "flagKey": "checkout-flow-v2",
        "variationId": "variation-a-id"
      }
    },
    {
      "name": "New Checkout",
      "baseline": false,
      "parameters": {
        "flagKey": "checkout-flow-v2",
        "variationId": "variation-b-id"
      }
    }
  ]
}

使用

create-experiment

工具，需提供：

```
projectKey
```
和
```
environmentKey
```
-- 实验运行的项目和环境
```
name
```
-- 实验的描述性名称
```
flagKey
```
-- 用于实验的功能标志键
```
metrics
```
-- 指标对象数组，包含
```
key
```
和
```
isGroup
```
字段
```
treatments
```
-- 变体数组，每个变体包含
```
name
```
、
```
baseline
```
标志和
```
parameters
```
```
holdout
```
（可选） -- 排除的流量百分比

json

{
  "projectKey": "my-project",
  "environmentKey": "production",
  "name": "Checkout Flow v2 Experiment",
  "flagKey": "checkout-flow-v2",
  "metrics": [
    {"key": "checkout-completed", "isGroup": false},
    {"key": "checkout-time-seconds", "isGroup": false}
  ],
  "treatments": [
    {
      "name": "Control",
      "baseline": true,
      "parameters": {
        "flagKey": "checkout-flow-v2",
        "variationId": "variation-a-id"
      }
    },
    {
      "name": "New Checkout",
      "baseline": false,
      "parameters": {
        "flagKey": "checkout-flow-v2",
        "variationId": "variation-b-id"
      }
    }
  ]
}

Step 3: Start Data Collection

步骤3：启动数据收集

Use

start-experiment-iteration

to begin collecting data:

json

{
  "projectKey": "my-project",
  "environmentKey": "production",
  "experimentKey": "checkout-flow-v2-experiment"
}

Optionally set

reshuffle: true

to redistribute traffic across treatments.

使用

start-experiment-iteration

开始收集数据：

json

{
  "projectKey": "my-project",
  "environmentKey": "production",
  "experimentKey": "checkout-flow-v2-experiment"
}

可选择设置

reshuffle: true

，在变体间重新分配流量。

Step 4: Verify

步骤4：验证

Use
```
get-experiment
```
to confirm the experiment is running
Check that all treatments are listed correctly
Verify metrics are attached
Confirm the iteration status shows as active

Report results:

Experiment created and iteration started
N treatments with M metrics configured
Data collection is active

使用
```
get-experiment
```
确认实验正在运行
检查所有变体是否正确列出
验证指标已关联
确认迭代状态显示为活跃

结果报告：

实验已创建且迭代已启动
已配置N个变体和M个指标
数据收集正在进行中

Edge Cases

边缘情况

Situation	Action
Metric doesn't exist	Create it first with `create-metric`
Flag has no variations	Create flag variations before setting up treatments
Experiment already exists	Use `list-experiments` to find it, then `get-experiment` for details
Need to change metrics mid-experiment	Stop the current iteration, update, then start a new one

场景	操作
指标不存在	先使用 `create-metric` 创建指标
标志无变体	设置变体前先创建功能标志变体
实验已存在	使用 `list-experiments` 找到实验，然后用 `get-experiment` 查看详情
实验中途需要修改指标	停止当前迭代，更新实验，然后启动新迭代

What NOT to Do

禁止操作

Don't start an experiment without clearly defined metrics
Don't stop experiments too early — wait for statistical significance
Don't run multiple experiments on the same flag simultaneously without careful holdout design
Don't forget to set a baseline treatment — one treatment must be marked
```
baseline: true
```

不要在没有明确定义指标的情况下启动实验
不要过早停止实验——等待统计显著性结果
不要在同一个功能标志上同时运行多个实验，除非有精心设计的保留组
不要忘记设置基线变体——必须有一个变体标记为
```
baseline: true
```