Back to Details

adversarial-review

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Adversarial review

对抗性评审

Static review and design councils miss bugs that only surface when code meets real input. This does the opposite: agents whose job is to break the code by driving the real system. On shux task 078 this pattern found a shipped-blocking bug (a validator that rejected ❤️ ⚠️ emoji) that both the author and three design councils had passed over.
静态评审和设计委员会往往会遗漏那些只有当代码处理真实输入时才会暴露的bug。本文介绍的方法则截然相反：让代理通过驱动真实系统来攻破代码。在shux任务078中，这种模式发现了一个阻碍发布的bug（某个验证器拒绝了❤️ ⚠️表情符号），而该bug被代码作者和三个设计委员会均忽略了。

When

适用时机

Standard step for any nontrivial feature / schema / contract / parser / serializer / protocol / state machine / security or freeze guard — after it compiles green, before the final dootsabha convergence review and before "done". Skip only for trivial or mechanical changes.
对于任何非 trivial 的功能/schema/契约/解析器/序列化器/协议/状态机/安全或冻结防护机制，这都是标准步骤——在代码编译通过（green）后、最终dootsabha收敛评审之前，以及标记“完成”之前。仅在处理 trivial 或机械性变更时可跳过此步骤。

Method

实施方法

  1. Split the attack surface into 2–4 disjoint areas (e.g. round-trip & canonicalization · the verdict/state model · the security/guard). One agent per area. Run them in parallel, in the background.
  2. Charter each agent (template below): its ONE surface, the PROMISE to falsify, 5–10 concrete hostile inputs, and the hard rule — drive the real system; do not reason from the source alone.
  3. Independently reproduce every finding before believing it. Agents over- and under-claim; a confident report is a hypothesis. Turn each real defect into a failing test first.
  4. Fix + pin. Every confirmed defect gets a fix AND a regression test (or a new proptest-generator case) so it cannot return.
  5. Feed forward. Record which vectors were verified clean and any out-of-scope findings for the owning task.
  1. 拆分攻击面：将攻击面划分为2-4个相互独立的区域（例如：往返与规范化 · 裁决/状态模型 · 安全/防护机制）。每个区域分配一个代理，在后台并行运行。
  2. 为每个代理分配任务（模板如下）：明确其负责的单一攻击面、需证伪的假设、5-10个具体的恶意输入，以及硬性规则——驱动真实系统；不得仅从代码源推理。
  3. 独立复现所有发现：在确认问题之前，必须独立复现每个发现。代理可能会夸大或低估问题，一份看似确凿的报告只是一个假设。首先将每个真实缺陷转化为失败测试用例。
  4. 修复并固化：每个已确认的缺陷都需修复，并添加回归测试（或新的proptest-generator用例），防止问题再次出现。
  5. 反馈迭代：记录哪些攻击向量已验证无问题，以及所有超出当前任务范围的发现，供相关任务参考。

Charter template

任务模板

ADVERSARIAL review. Read-only; report findings, do not edit. Target: 
<file / surface>
. Drive the REAL system — build a scratch harness with a path-dep, or run the real binary, and feed it hostile input. Do NOT reason from source alone; that is where the sharpest bugs hide. PROMISE to falsify: 
<the invariant — e.g. "any real input round-trips losslessly", "exactly one canonical encoding", "masked content never leaks", "the guard cannot be bypassed">
. Attack vectors — for each, say REAL BUG or CORRECTLY HANDLED, with 
file:line
, the exact triggering input, and whether an existing test catches it: 
<5–10 concrete hostile inputs>
. Rank BLOCKER / MAJOR / MINOR. If the code is correct on an attack, say so in one line. End with a ranked defect list + a concrete failing test for the top 3.
对抗性评审。仅可读；报告发现，请勿编辑。目标：
<文件/攻击面>
。 驱动真实系统——构建包含路径依赖的临时测试工具，或运行真实二进制文件，向其输入恶意数据。不得仅从代码源推理；最隐蔽的bug正藏于此。 需证伪的假设：
<不变量——例如“任何真实输入均可无损往返”“仅存在一种规范化编码”“掩码内容绝不会泄露”“防护机制无法被绕过”>
。 攻击向量——针对每个向量，标注“真实bug”或“处理正确”，并提供
文件:行号
、确切触发输入，以及现有测试是否能覆盖该情况：
<5-10个具体的恶意输入>
。 按BLOCKER/MAJOR/MINOR分级。若代码成功抵御某次攻击，用一句话说明即可。 结尾附上按优先级排序的缺陷列表，以及排名前三的具体失败测试用例。

Discipline

执行准则

  • Real system > reasoning. The blocker on 078 came from an agent that RAN the real VT on 
    ❤️
    ; static reasoning (and the author) dismissed the same vector as theoretical.
  • Your own self-checks are not a substitute. Expect to be wrong about your own code — that is the entire point of the pass.
  • Verify before fixing. Reproduce independently; downgrade what you can't.
  • 真实系统优先于推理：任务078中的阻碍性bug来自一个运行真实VT处理
    ❤️
    的代理；而静态推理（包括代码作者）将同一攻击向量视为理论性问题而忽略。
  • 自我检查无法替代此步骤：要接受自己对代码的判断可能有误——这正是此步骤的核心意义。
  • 先验证再修复：独立复现问题；对于无法复现的问题降低优先级。

Anti-patterns

反模式

  • Overlapping charters → agents duplicate work and leave gaps. Keep areas disjoint.
  • "Review this file" → invites static reading. Always say drive the real system.
  • One mega-agent → weaker than N focused parallel ones.
  • Trusting a finding without an independent repro, or fixing without a regression test.
  • 任务重叠 → 代理重复工作且存在遗漏。确保各区域相互独立。
  • "评审此文件" → 易导致静态阅读。务必强调驱动真实系统。
  • 单一大型代理 → 效果弱于N个专注的并行代理。
  • 未独立复现就信任发现，或修复时未添加回归测试。