🇺🇸

Original

English

🇨🇳

Translation

Chinese

Experiment Results

实验结果

An experiment results document captures what happened when you tested a hypothesis, including statistical outcomes, segment analysis, learnings, and clear recommendations. Good results documentation turns individual experiments into organizational knowledge that improves future decision-making.

实验结果文档记录了验证假设过程中的实际情况，包括统计结果、细分群体分析、经验总结以及明确的建议。优质的结果文档能将单个实验转化为组织知识，助力未来的决策制定。

When to Use

使用场景

After an A/B test or experiment reaches statistical significance
When an experiment is ended early (for any reason)
To communicate findings to stakeholders who weren't involved
During decision-making about whether to ship, iterate, or kill a feature
To build a repository of learnings that inform future experiments

在A/B测试或实验达到统计显著性之后
实验提前终止时（无论原因是什么）
向未参与实验的利益相关者沟通研究发现时
在决定是否发布、迭代或终止某个功能的决策过程中
建立经验知识库，为未来的实验提供参考

Instructions

撰写步骤

When asked to document experiment results, follow these steps:

Summarize the Experiment Provide context: what was tested, when it ran, how much traffic it received. Link to the original experiment design document if one exists.
Restate the Hypothesis Remind readers what you believed would happen and why. This frames the results interpretation.
Present Primary Results Show the primary metric outcome clearly: what were the values for control and treatment? Include statistical significance (p-value), confidence intervals, and sample sizes. Be honest about whether results are conclusive.
Analyze Secondary Metrics Present guardrail metrics that ensure you didn't cause unintended harm. Note any secondary metrics that moved unexpectedly.both positive and negative.
Segment the Data Look for differential effects across user segments (platform, tenure, plan type, etc.). Sometimes overall results mask important segment-level insights.
Extract Learnings What did you learn beyond the numbers? Include surprising findings, questions raised, and implications for the product hypothesis. Negative results are valuable learnings.
Make a Recommendation Be clear: should we ship, iterate, or kill? Support the recommendation with the evidence. If the decision is nuanced, explain the trade-offs.
Define Next Steps Specify what happens now.engineering work to ship, follow-up experiments, metrics to continue monitoring, or documentation to update.

当需要记录实验结果时，请遵循以下步骤：

总结实验概况 提供背景信息：测试内容、运行时间、流量规模。如有原始实验设计文档，请附上链接。
重述假设 提醒读者你原本预期会发生什么以及原因。这有助于框定结果的解读方向。
呈现核心结果 清晰展示核心指标的结果：对照组和实验组的数值分别是多少？包含统计显著性（p值）、置信区间和样本量。如实说明结果是否具有结论性。
分析次要指标 展示保障指标，确保未造成意外影响。记录所有出现异常变动的次要指标，无论正向还是负向。
细分数据分析 观察不同用户群体（平台、使用时长、套餐类型等）的差异化影响。有时整体结果会掩盖重要的细分群体层面的洞察。
提炼经验总结 除了数据之外，你还学到了什么？包括意外发现、引发的问题以及对产品假设的启示。负面结果也是宝贵的经验。
给出建议 明确说明：我们应该发布、迭代还是终止？用证据支撑建议。如果决策存在细微差别，请解释利弊权衡。
确定后续步骤 明确当前的后续行动：发布所需的工程工作、后续实验、需要持续监控的指标，或需要更新的文档。

Output Format

输出格式

Use the template in

references/TEMPLATE.md

to structure the output.

使用

references/TEMPLATE.md

中的模板来构建输出内容。

Quality Checklist

质量检查清单

Before finalizing, verify:

Statistical methods and significance are clearly stated
Confidence intervals are included (not just p-values)
Segment analysis checked for differential effects
Secondary/guardrail metrics are reported
Learnings go beyond just the numbers
Recommendation is clear and actionable
Negative or inconclusive results are reported honestly

定稿前，请验证以下内容：

统计方法和显著性已清晰说明
已包含置信区间（而非仅p值）
已检查细分群体分析的差异化影响
已报告次要/保障指标
经验总结不局限于数据本身
建议清晰且可执行
如实报告负面或无结论的结果

See

references/EXAMPLE.md

for a completed example.

请查看

references/EXAMPLE.md

中的完整示例。

measure-experiment-results