Back to Details

diagnosing-bugs

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Diagnosing Bugs

Diagnosing Bugs

面向棘手 bugs 的纪律。只有在明确说明理由时才跳过阶段。
探索 codebase 时，先读取 
CONTEXT.md
（如果存在），建立相关 modules 的清晰 mental model，并检查你将触碰区域的 ADRs。
Discipline for tricky bugs. Only skip phases if you explicitly state why.
When exploring the codebase, first read 
CONTEXT.md
(if it exists) to build a clear mental model of relevant modules, and check the ADRs for the areas you will touch.

Phase 1 - Build a feedback loop

Phase 1 - Build a feedback loop

这就是这个 skill 的核心。 其他所有内容都是机械步骤。如果你拥有一个针对该 bug 的 tight pass/fail signal，即它会在 这个 bug 上变红，你就能找到原因；bisection、hypothesis-testing 和 instrumentation 都只是消费这个 signal。没有它，盯着代码看多久都救不了你。
在这里投入不成比例的精力。要强硬、要有创造力、拒绝放弃。
This is the core of this skill. Everything else is mechanical steps. If you have a tight pass/fail signal for the bug—one that turns red for this bug—you can find the cause; bisection, hypothesis-testing, and instrumentation all just consume this signal. Without it, staring at code for hours won’t save you.
Invest disproportionate effort here. Be tough, be creative, refuse to give up.

Ways to construct one - try them in roughly this order

Ways to construct one - try them in roughly this order

  1. Failing test，放在能触达 bug 的 seam 上：unit、integration、e2e 都可以。
  2. Curl / HTTP script，打到运行中的 dev server。
  3. CLI invocation，使用 fixture input，并把 stdout 与 known-good snapshot diff。
  4. Headless browser script（Playwright / Puppeteer），驱动 UI，并断言 DOM/console/network。
  5. Replay a captured trace. 把真实 network request / payload / event log 保存到磁盘，并在隔离环境中 replay 到代码路径。
  6. Throwaway harness. 启动系统的最小子集（一个 service、mocked deps），用一次 function call 触发 bug code path。
  7. Property / fuzz loop. 如果 bug 是 "sometimes wrong output"，运行 1000 个 random inputs 并寻找 failure mode。
  8. Bisection harness. 如果 bug 出现在两个已知状态之间（commit、dataset、version），自动化 "boot at state X, check, repeat"，这样可以 
    git bisect run
    。
  9. Differential loop. 用同一 input 跑 old-version vs new-version（或两个 configs），然后 diff outputs。
  10. HITL bash script. 最后手段。如果必须由人点击，就用 
    scripts/hitl-loop.template.sh
    驱动 人，让 loop 仍保持结构化。捕获的输出反馈给你。
构建正确的 feedback loop，bug 就修好了 90%。
  1. Failing test, placed at a seam that reaches the bug: unit, integration, e2e are all acceptable.
  2. Curl / HTTP script, hitting a running dev server.
  3. CLI invocation, using fixture input, and diff stdout against a known-good snapshot.
  4. Headless browser script (Playwright / Puppeteer), driving the UI, and asserting on DOM/console/network.
  5. Replay a captured trace. Save real network requests / payloads / event logs to disk, and replay them into the code path in an isolated environment.
  6. Throwaway harness. Start the minimal subset of the system (one service, mocked dependencies), and trigger the bug code path with a single function call.
  7. Property / fuzz loop. If the bug is "sometimes wrong output", run 1000 random inputs and look for failure modes.
  8. Bisection harness. If the bug appears between two known states (commit, dataset, version), automate "boot at state X, check, repeat" so you can use 
    git bisect run
    .
  9. Differential loop. Run old-version vs new-version (or two configurations) with the same input, then diff the outputs.
  10. HITL bash script. Last resort. If a human must click, use 
    scripts/hitl-loop.template.sh
    to drive the human, keeping the loop structured. Captured output feeds back to you.
Build the right feedback loop, and the bug is 90% fixed.

Tighten the loop

Tighten the loop

把 loop 当作产品。只要有了 一个 loop，就继续 tighten 它：
  • 我能让它更快吗？（Cache setup、跳过无关 init、缩小 test scope。）
  • 我能让 signal 更尖锐吗？（断言具体 symptom，而不是 "didn't crash"。）
  • 我能让它更 deterministic 吗？（Pin time、seed RNG、isolate filesystem、freeze network。）
一个 30 秒且 flaky 的 loop 几乎不比没有 loop 好；一个 2 秒 deterministic loop 才是 tight 的调试超能力。
Treat the loop like a product. Once you have one loop, keep tightening it:
  • Can I make it faster? (Cache setup, skip irrelevant initialization, narrow test scope.)
  • Can I make the signal sharper? (Assert the specific symptom, not "didn't crash".)
  • Can I make it more deterministic? (Pin time, seed RNG, isolate filesystem, freeze network.)
A 30-second flaky loop is barely better than no loop; a 2-second deterministic loop is a tight debugging superpower.

Non-deterministic bugs

Non-deterministic bugs

目标不是 clean repro，而是 higher reproduction rate。循环触发 100x、parallelise、加 stress、缩小 timing windows、注入 sleeps。50%-flake bug 可以调试；1% 不行。持续提高复现率，直到它可调试。
The goal isn’t a clean repro, but higher reproduction rate. Trigger the loop 100x, parallelize, add stress, narrow timing windows, inject sleeps. A 50%-flake bug can be debugged; 1% cannot. Keep increasing the reproduction rate until it’s debuggable.

When you genuinely cannot build a loop

When you genuinely cannot build a loop

停下来并明确说明。列出你尝试过什么。向用户请求：(a) 能复现的环境访问权限，(b) 捕获的 artifact（HAR file、log dump、core dump、带 timestamps 的 screen recording），或 (c) 添加临时 production instrumentation 的许可。不要 在没有 loop 时继续 hypothesise。
Stop and state this explicitly. List what you’ve tried. Ask the user for: (a) access to a reproducible environment, (b) captured artifacts (HAR file, log dump, core dump, timestamped screen recording), or (c) permission to add temporary production instrumentation. Do not continue hypothesizing without a loop.

Completion criterion - a tight loop that goes red

Completion criterion - a tight loop that goes red

Phase 1 完成条件：loop tight 且 red-capable。你能指出 一个 command（script path、test invocation、curl），并且你已经至少运行过一次（贴出 invocation 和 output），且它满足：
  • Red-capable - 它驱动真实 bug code path，并断言 用户的 exact symptom，因此能在该 bug 上变红、修复后变绿。不是 "runs without erroring"，而是必须能 catch this specific bug。
  • Deterministic - 每次运行 verdict 相同（flaky bugs：按上文固定到高复现率）。
  • Fast - 秒级，而不是分钟级。
  • Agent-runnable - 你可以无人值守运行；human in the loop 只能通过 
    scripts/hitl-loop.template.sh
    。
如果你发现自己在 command 存在前就读代码构建理论，停下；直接跳到 hypothesis 正是这个 skill 要防止的失败。 没有 red-capable command，就没有 Phase 2。
Phase 1 completion criteria: the loop is tight and red-capable. You can point to one command (script path, test invocation, curl), and you’ve run it at least once (post the invocation and output), and it meets:
  • Red-capable - It drives the real bug code path, and asserts the user's exact symptom, so it turns red for this bug and green once fixed. Not "runs without erroring", but must catch this specific bug.
  • Deterministic - Verdict is the same every run (for flaky bugs: fix to high reproduction rate as above).
  • Fast - Seconds, not minutes.
  • Agent-runnable - You can run it unattended; human-in-the-loop is only allowed via 
    scripts/hitl-loop.template.sh
    .
If you find yourself reading code to build theories before the command exists, stop; jumping straight to hypothesis is the failure this skill is designed to prevent. Without a red-capable command, there is no Phase 2.

Phase 2 - Reproduce + minimise

Phase 2 - Reproduce + minimise

运行 loop。看它变红，也就是 bug 出现。
确认：
  • Loop 产出的 failure mode 是 用户 描述的那个，而不是附近另一个失败。Wrong bug = wrong fix。
  • Failure 能在多次运行中复现（或对于 non-deterministic bugs，复现率足够高，能用来调试）。
  • 你已捕获 exact symptom（error message、wrong output、slow timing），后续阶段可以验证 fix 确实解决它。
Run the loop. Watch it turn red—i.e., the bug appears.
Confirm:
  • The failure mode produced by the loop is the exact one described by the user, not another nearby failure. Wrong bug = wrong fix.
  • The failure can be reproduced across multiple runs (or for non-deterministic bugs, the reproduction rate is high enough to use for debugging).
  • You’ve captured the exact symptom (error message, wrong output, slow timing), so subsequent phases can verify the fix actually resolves it.

Minimise

Minimise

一旦变红，就把 repro 缩到 仍会变红的最小场景。逐个削减 inputs、callers、config、data 和 steps，每次削减后重新运行 loop；只保留 failure 的 load-bearing 部分。
原因：minimal repro 会缩小 Phase 3 的 hypothesis space（可怀疑的 moving parts 更少），并成为 Phase 5 中干净的 regression test。
完成条件：每个剩余元素都是 load-bearing，移除任意一个都会让 loop 变绿。
在 reproduce 并 minimise 之前不要继续。
Once it’s red, shrink the repro to the smallest possible scenario that still turns red. Cut inputs, callers, config, data, and steps one by one, re-running the loop after each cut; only keep the load-bearing parts of the failure.
Why: A minimal repro narrows the hypothesis space for Phase 3 (fewer moving parts to suspect), and becomes a clean regression test in Phase 5.
Completion criterion: Every remaining element is load-bearing—removing any of them makes the loop turn green.
Do not proceed until you’ve reproduced and minimised.

Phase 3 - Hypothesise

Phase 3 - Hypothesise

在测试任何假设前，生成 3-5 个 ranked hypotheses。单假设会锚定在第一个看似合理的想法上。
每个 hypothesis 必须 falsifiable：说明它会做出什么 prediction。
Format: "If <X> is the cause, then <changing Y> will make the bug disappear / <changing Z> will make it worse."
如果无法说明 prediction，这就是 vibe；丢弃或打磨它。
测试前把 ranked list 展示给用户。 用户常常有 domain knowledge，可以立即重排（"we just deployed a change to #3"），或知道哪些 hypotheses 已被排除。便宜 checkpoint，大幅省时。不要因此阻塞；如果用户 AFK，就按你的排序继续。
Before testing any hypotheses, generate 3-5 ranked hypotheses. A single hypothesis anchors you to the first plausible idea.
Each hypothesis must be falsifiable: state what prediction it makes.
Format: "If <X> is the cause, then <changing Y> will make the bug disappear / <changing Z> will make it worse."
If you can’t state the prediction, it’s a vibe; discard or refine it.
Show the ranked list to the user before testing. Users often have domain knowledge that can immediately reorder it ("we just deployed a change to #3"), or know which hypotheses have already been ruled out. Cheap checkpoint that saves massive time. Don’t block on this; if the user is AFK, proceed with your ranking.

Phase 4 - Instrument

Phase 4 - Instrument

每个 probe 都必须映射到 Phase 3 的某个具体 prediction。一次只改变一个变量。
Tool preference：
  1. Debugger / REPL inspection，如果环境支持。一个 breakpoint 胜过十条 logs。
  2. Targeted logs，放在能区分 hypotheses 的 boundaries。
  3. 永远不要 "log everything and grep"。
给每条 debug log 加唯一 prefix，例如 
[DEBUG-a4f2]
。最后 cleanup 就能一次 grep。未打 tag 的 logs 会存活；带 tag 的 logs 要删除。
Perf branch。 对 performance regressions，logs 通常不对。改为先建立 baseline measurement（timing harness、
performance.now()
、profiler、query plan），然后 bisect。先 measure，再 fix。
Each probe must map to a specific prediction from Phase 3. Change only one variable at a time.
Tool preference:
  1. Debugger / REPL inspection, if the environment supports it. One breakpoint is worth ten logs.
  2. Targeted logs, placed at boundaries that distinguish hypotheses.
  3. Never "log everything and grep".
Add a unique prefix to each debug log, e.g., 
[DEBUG-a4f2]
. Cleanup at the end can grep them all at once. Untagged logs will linger; tagged logs must be deleted.
Perf branch. For performance regressions, logs are usually not helpful. Instead, first establish a baseline measurement (timing harness, 
performance.now()
, profiler, query plan), then bisect. Measure first, then fix.

Phase 5 - Fix + regression test

Phase 5 - Fix + regression test

在 fix 前写 regression test，但前提是存在 correct seam。
Correct seam 是 test 能以 call site 中真实发生的方式触发 real bug pattern 的地方。如果唯一可用 seam 太 shallow（bug 需要多个 callers，但 test 只有 single-caller；unit test 无法复制触发 bug 的 chain），那里的 regression test 会给出 false confidence。
如果不存在 correct seam，这本身就是发现。 记录下来。Codebase architecture 阻止你锁住 bug。把它标记给下一阶段。
如果存在 correct seam：
  1. 把 minimised repro 变成该 seam 上的 failing test。
  2. 看它 fail。
  3. 应用 fix。
  4. 看它 pass。
  5. 重新针对原始（未 minimised）场景运行 Phase 1 feedback loop。
Write the regression test before fixing, but only if a correct seam exists.
A correct seam is where the test can trigger the real bug pattern in the way it actually occurs at the call site. If the only available seam is too shallow (the bug requires multiple callers, but the test has only a single caller; a unit test can’t replicate the chain that triggers the bug), the regression test there will give false confidence.
If no correct seam exists, this is a finding in itself. Document it. The codebase architecture prevents you from locking in the bug. Flag it for the next phase.
If a correct seam exists:
  1. Turn the minimised repro into a failing test at that seam.
  2. Watch it fail.
  3. Apply the fix.
  4. Watch it pass.
  5. Re-run the Phase 1 feedback loop against the original (unminimised) scenario.

Phase 6 - Cleanup + post-mortem

Phase 6 - Cleanup + post-mortem

声明完成前必须做：
  • Original repro 不再复现（重跑 Phase 1 loop）
  • Regression test 通过（或记录缺少 seam）
  • 所有 
    [DEBUG-...]
    instrumentation 已移除（grep prefix）
  • Throwaway prototypes 已删除（或移动到明确标记的 debug location）
  • 正确 hypothesis 已写进 commit / PR message，让下一个 debugger 能学习
然后问：什么本可以预防这个 bug？ 如果答案涉及 architecture change（没有好 test seam、callers 缠绕、hidden coupling），带着具体细节交给 
/improve-codebase-architecture
skill。这个建议要在 fix 之后提出，不要在之前提出；现在你比开始时知道得更多。
Must do before declaring completion:
  • Original repro no longer reproduces (re-run Phase 1 loop)
  • Regression test passes (or document the missing seam)
  • All 
    [DEBUG-...]
    instrumentation has been removed (grep the prefix)
  • Throwaway prototypes have been deleted (or moved to a clearly marked debug location)
  • The correct hypothesis has been written into the commit / PR message, so the next debugger can learn
Then ask: What could have prevented this bug? If the answer involves architecture changes (no good test seam, tangled callers, hidden coupling), pass this to the 
/improve-codebase-architecture
skill with specific details. This suggestion should come after the fix, not before; you know much more now than you did at the start.