Back to Details

causal-inference

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Causal Inference

Causal Inference（因果推断）

Core principle: Correlation is not causation — but sometimes it is, and knowing which matters enormously. Use counterfactuals, confounders, and causal structure to ask "did X actually cause Y?" rigorously before acting on data.
核心原则：Correlation并非causation——但有时二者确实存在因果关系，区分二者至关重要。在依据数据采取行动前，需借助反事实推理、混杂因素和因果结构严谨地探究“X是否真的导致了Y？”

The Core Distinction

核心区别

Correlation: X and Y move together. Causation: Changing X changes Y — and we know why.
Why it matters:
  • Intervening on a correlate with no causal path wastes effort
  • Missing a confounder leads to attributing effects to the wrong cause
  • Acting on spurious correlation can make things worse
Correlation：X与Y同步变化。 Causation：改变X会改变Y——并且我们知道其中的原因。
为什么这很重要：
  • 对不存在因果路径的相关因素进行干预会白费力气
  • 遗漏混杂因素会导致将效果错误归因于无关原因
  • 基于虚假相关性采取行动可能会使情况恶化

Key Concepts

关键概念

Counterfactual Reasoning

Counterfactual Reasoning（反事实推理）

The fundamental question:
"What would have happened to Y if X had been different, all else equal?"
You never observe both the treated and untreated state of the same unit at the same time — the fundamental problem of causal inference. Every causal claim is implicitly counterfactual; make it explicit.
核心问题：
“如果X有所不同，其他条件不变，Y会发生什么变化？”
你永远无法同时观察同一主体在同一时间的处理状态和未处理状态——这是causal inference的核心难题。每一个因果论断都隐含反事实假设；需将其明确化。

Confounders

Confounders（混杂因素）

A third variable Z that causally affects both X and Y, creating correlation between them without a direct causal path.
Z → X
Z → Y
X and Y correlate, but X doesn't cause Y. Intervening on X does nothing.
Example: Ice cream sales and drowning rates correlate. Confounder: hot weather → more ice cream AND more swimming → more drowning. Banning ice cream doesn't reduce drowning.
Common confounders in product/engineering work:
  • Seasonality (feature adoption and engagement move together)
  • Selection bias (users who adopt are already more engaged)
  • External events (a competitor shut down the same week you shipped)
  • Time trends (both metrics were already moving before intervention)
一个同时对X和Y产生因果影响的第三变量Z，会在X和Y之间建立相关性，但二者并无直接因果路径。
Z → X
Z → Y
X与Y相关，但X并不导致Y。对X进行干预不会产生任何效果。
示例：冰淇淋销量和溺水率相关。混杂因素：高温天气→冰淇淋销量增加，同时游泳人数增多→溺水事件增加。禁止冰淇淋无法降低溺水率。
产品/工程工作中的常见混杂因素：
  • 季节性（功能采用率和用户活跃度同步变化）
  • 选择偏差（采用功能的用户原本就更活跃）
  • 外部事件（你发布功能的同一周，竞争对手倒闭了）
  • 时间趋势（在干预前，两个指标就已经呈现变动趋势）

Mediators vs. Confounders

中介因素 vs. 混杂因素

A mediator is on the causal path — X → M → Y. Blocking it blocks the effect. A confounder is upstream of both — control for it.
Confusing them causes overcorrection (controlling for a mediator removes the effect you're looking for).
中介因素位于因果路径上——X → M → Y。阻断中介因素会阻断因果效应。 混杂因素位于二者的上游——需对其进行控制。
混淆二者会导致过度校正（控制中介因素会消除你所关注的效果）。

Simpson's Paradox

Simpson's Paradox（辛普森悖论）

An observed trend can reverse when data is aggregated. A treatment can appear harmful in aggregate but beneficial in every subgroup (or vice versa) due to unequal group sizes.
Always ask: Does disaggregating change the conclusion?
当数据聚合时，观察到的趋势可能会逆转。由于群体规模不均，某一处理在整体数据中看似有害，但在每个子群体中却有益（反之亦然）。
务必询问：拆分数据后结论是否发生变化？

Tools for Establishing Causation

确立因果关系的工具

Randomized Controlled Experiment (Gold Standard)

随机对照试验（黄金标准）

Random assignment eliminates confounding by making treatment independent of all other variables.
In product work: A/B tests are RCTs. Validity depends on:
  • Random assignment (not self-selection)
  • Sufficient sample size (statistical power)
  • Single treatment change (no simultaneous changes)
  • No interference between units (SUTVA)
  • Correct metric selection
A/B test failure modes:
  • Novelty effect: early lift decays as users habituate
  • Sample Ratio Mismatch: unequal group sizes indicating randomization failure
  • Multiple comparisons: 20 metrics gives 1 false positive by chance at p=0.05
  • Peeking: stopping early when results look good inflates false positive rate
随机分配通过使处理与所有其他变量独立，消除了混杂因素的影响。
在产品工作中：A/B测试就是RCT。其有效性取决于：
  • 随机分配（而非自我选择）
  • 足够的样本量（统计功效）
  • 单一处理变化（无同步变更）
  • 主体间无干扰（SUTVA）
  • 正确的指标选择
A/B测试失效模式：
  • 新奇效应：初期指标提升会随着用户习惯化而衰减
  • 样本比例不匹配：群体规模不均表明随机化失败
  • 多重比较：在p=0.05的显著性水平下，20个指标中会有1个出现假阳性结果
  • 提前偷看：当结果看似良好时提前终止测试会提高假阳性率

Difference-in-Differences (DiD)

双重差分法（DiD）

Compare the change for a treated group vs. control over time.
Effect = (Treated_after - Treated_before) - (Control_after - Control_before)
Assumes: Without treatment, both groups would have followed parallel trends. Use when: You have pre/post data and a natural control group but couldn't randomize.
比较处理组与对照组随时间的变化差异。
Effect = (Treated_after - Treated_before) - (Control_after - Control_before)
假设前提：若未进行处理，两组的趋势会保持平行。 适用场景：你拥有前后数据和自然对照组，但无法进行随机化。

Natural Experiments

自然实验

External factors create quasi-random treatment variation — policy changes, geographic boundaries, system outages, cohort-based rollouts.
Example: Feature rolled out by sign-up date — early users are treatment, later users are control (if no self-selection in timing).
外部因素创造了准随机的处理差异——政策变化、地理边界、系统故障、基于队列的发布。
示例：按注册日期发布功能——早期用户为处理组，后期用户为对照组（若发布时机无自我选择偏差）。

Causal Graph (DAG)

因果图（DAG）

Map all variables and their causal relationships. Makes confounders and mediators explicit and determines what to control for.
[Confounder Z] → [Treatment X] → [Mediator M] → [Outcome Y]
      ↓___________________________________↑
Reading the DAG: control for Z (confounder), don't control for M (mediator).
绘制所有变量及其因果关系。明确混杂因素和中介因素，并确定需要控制的变量。
[Confounder Z] → [Treatment X] → [Mediator M] → [Outcome Y]
      ↓___________________________________↑
解读DAG：控制Z（混杂因素），不要控制M（中介因素）。

Output Format

输出格式

🔍 Causal Claim Under Examination

🔍 待验证的因果论断

  • Stated claim: [What is asserted to cause what]
  • Reformulated as counterfactual: "Would Y have been different if X had not occurred, all else equal?"
  • 表述的论断：[断言的因果关系是什么]
  • 重构为反事实假设：“如果X未发生，其他条件不变，Y会有所不同吗？”

🕸️ Causal Structure

🕸️ 因果结构

Sketch the causal graph:
  • Proposed causal paths?
  • Potential confounders?
  • Mediators (on the causal path)?
  • Colliders (caused by both X and Y — controlling opens spurious paths)?
绘制因果图：
  • 提出的因果路径？
  • 潜在的混杂因素？
  • 中介因素（位于因果路径上）？
  • 对撞因素（由X和Y共同导致——控制该因素会开启虚假路径）？

⚠️ Threats to Causal Interpretation

⚠️ 因果解读的威胁

For each: Present / Possible / Unlikely
ThreatPresent?EvidenceImpact on Conclusion
Confounding
Selection bias
Reverse causation (Y → X)
Common cause (Z → X, Z → Y)
Seasonality / time trend
Coincidental timing
Simpson's Paradox
针对每项威胁：存在/可能存在/不太可能存在
威胁是否存在？证据对结论的影响
混杂因素
选择偏差
反向因果（Y → X）
共同原因（Z → X, Z → Y）
季节性/时间趋势
巧合性时机
辛普森悖论

📊 Evidence Quality

📊 证据质量

  • Design used: [RCT / DiD / Natural experiment / Observational]
  • Evidence strength: [Strong / Moderate / Weak]
  • Key assumptions: [What must be true for the design to be valid]
  • Assumption violations: [Any signs assumptions don't hold]
  • 采用的设计：[RCT / DiD / 自然实验 / 观测性研究]
  • 证据强度：[强 / 中等 / 弱]
  • 关键假设：[设计有效必须满足的前提条件]
  • 假设违背：[任何表明假设不成立的迹象]

🎯 Conclusion

🎯 结论

  • Causal claim warranted?: [Yes / Probably / Unclear / No]
  • If yes: Estimated effect size and confidence
  • If unclear: What evidence would resolve it?
  • If no: What alternative explanation better fits the data?
  • 因果论断是否成立？：[是 / 可能是 / 不确定 / 否]
  • 如果成立：估计的效应量和置信度
  • 如果不确定：哪些证据可以解决疑问？
  • 如果不成立：哪种替代解释更符合数据？

🔬 Next Steps

🔬 下一步行动

  • What experiment would establish causation most efficiently?
  • What natural variation in the data could be exploited?
  • What confounders should be measured and controlled for?
  • 哪种实验能最有效地确立因果关系？
  • 数据中存在哪些可利用的自然差异？
  • 应测量并控制哪些混杂因素？

Causal Inference Checklist for A/B Tests

A/B测试的因果推断检查清单

Before trusting a result:
  • Was assignment truly random? Check Sample Ratio Mismatch.
  • Was only one thing changed?
  • Is sample size sufficient for the expected effect?
  • Was the test run for a full weekly cycle?
  • Is the primary metric pre-specified?
  • Do secondary metrics that should move actually move?
  • Is there a plausible mechanism explaining why X would cause Y?
  • Is the effect consistent across segments? (Check Simpson's Paradox)
在信任测试结果前：
  • 分配是否真正随机？检查样本比例不匹配情况。
  • 是否仅变更了一项内容？
  • 样本量是否足以检测预期效应？
  • 测试是否运行了完整的一周周期？
  • 主要指标是否预先指定？
  • 预期会变动的次要指标是否真的变动了？
  • 是否存在合理的机制可以解释X为何会导致Y？
  • 效应在不同细分群体中是否一致？（检查辛普森悖论）

Thinking Triggers

思考触发点

  • "What's the counterfactual? What would have happened without this change?"
  • "What else changed at the same time that could explain this?"
  • "Are the units we're comparing actually comparable?"
  • "Is there a third variable that could explain the correlation?"
  • "Does the mechanism make sense — why would X cause Y?"
  • "Does disaggregating the data change the conclusion?"
  • "Would we see the same result if we ran this experiment again?"
  • “反事实情况是什么？如果没有这项变化，会发生什么？”
  • “同一时间还有哪些其他变化可以解释这一结果？”
  • “我们比较的主体是否真的具有可比性？”
  • “是否存在可以解释这种相关性的第三变量？”
  • “机制是否合理——X为什么会导致Y？”
  • “拆分数据后结论是否发生变化？”
  • “如果再次运行该实验，我们会得到相同的结果吗？”