referee-sim

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

referee-sim — the frame linter

referee-sim — frame linter（框架审查器）

Fact-checking and frame-checking are different audits. A verification pass confirms every sentence is true; this pass checks what a reader concludes. A document whose every claim is sourced can still mislead by assembly: the caveats live in the body while impressions form in the abstract; the comparison is honest but the baseline is one nobody uses; the theorem is real but the reader leaves believing it covers the case it excludes. That exact failure can survive multiple honesty passes and surface only when an outside expert finally reads the document — the expensive way to catch it, because by then the impression has already formed in the one reader whose report matters.

This is an audit of the IMPLICATURE — what each kind of reader walks away believing — run before anyone outside sees the document. Run your sentence-level style and honesty linter separately; the two passes catch disjoint failures, and neither substitutes for the other.

事实核查与框架核查是两种不同的审查。核验流程确认每个句子都为真；本流程检查读者会得出什么结论。一份每个论断都有来源的文档，仍可能因组合方式而造成误导：限定条件留在正文中，印象却在摘要中形成；比较是诚实的，但基线无人使用；定理确实成立，但读者离开时却认为它涵盖了自己实际上排除的情形。这种失败恰恰可能在多轮诚信审查后存活下来，直到外部专家最终阅读文档时才暴露——这是一种代价高昂的发现方式，因为此时印象已经在那位其意见至关重要的读者心中形成。

这是一项针对“隐含推论”的审查——每类读者最终会相信什么——须在任何外部人士看到文档之前运行。请将句级风格和诚信 linter 单独运行；这两轮检查发现的是互不重叠的失败，任何一轮都不能替代另一轮。

Procedure

Procedure（流程）

1. Enumerate the audiences, explicitly and in writing

1. 明确且书面列出所有受众

List every community that could plausibly write a report on this document. Sources for the list, in order:

The introduction's "this matters to X, Y, and Z" sentence. That sentence is a contract; every community it names gets an audit row.
Every community whose methods the document borrows. Using their machinery invokes their standards, whether or not the document ever addresses them.
Every community whose results the document takes as input or uses as a comparison.
The incumbent: whoever currently does the thing the document claims to do better, faster, or differently. This audience exists even when the introduction never mentions them — especially then.
The practitioner who would act on the result, whose question is never "is it interesting" but "what breaks if I rely on this."
The venue's general reader, who determines which claims must survive without the surrounding expertise.

Two rules. First, any community borrowed from for authority — a method, a dataset, a benchmark, a motivating application — gets a row without exception; the missing-audience failure below is almost always one of these. Second, a claim of interdisciplinarity RAISES the bar. Each additional audience brings its own standard objection, and outsiders read less charitably than insiders: they apply their home field's first-order standard and have no reason to extend the benefit of the doubt.

列出所有可能合理为本文件撰写报告的社群。名单按以下顺序确定：

引言中的“这对 X、Y 和 Z 很重要”那句话。该句是一份契约；其中点名的每个社群都必须获得一行审查记录。
文档借用了其方法的每个社群。采用他们的方法体系，就等于援引他们的标准，无论文档是否明确讨论这些标准。
文档将其结果作为输入或比较对象的每个社群。
在位方案（incumbent）：当前正在执行文档所声称能做得更好、更快或以不同方式完成之事的任何一方。即使引言从未提及他们，这个受众也依然存在——尤其是在引言未提时。
会依据结果采取行动的实践者；他们的问题从来不是“这是否有趣”，而是“如果我依赖它，会出什么问题？”
发表平台的一般读者；他们决定了哪些论断必须在缺少周边专业知识时仍然站得住脚。

两条规则。第一，任何被借用来提供权威依据的社群——一种方法、一套数据集、一项基准测试或一个启发式应用——都无一例外地获得一行记录；下文的“受众遗漏”失败几乎总是这一类。第二，跨学科声明会提高门槛。每增加一个受众，就会增加其自身的标准反对意见；外部人士比内部人士更少给予善意的解读：他们会套用本领域的一阶标准，而且没有理由给予额外信任或善意解释。

2. Generate each audience's strongest STANDARD objection — on two axes

2. 为每个受众生成最强的标准反对意见——沿两个维度

For each audience row, generate two separate objections:

Correctness: "is this right, and does it hold in the cases my community cares about?"
Novelty: "is this new, and what exactly is the advance over what we already do?"

These are different referee reports, answered in different places — correctness in scope statements, controls, and comparisons; novelty in the introduction's positioning and its engagement with prior work. A document that answers only one axis dies on the other, and conflating them produces a characteristic hybrid: a document that proves everything and never says what is new, or one that claims a first while the incumbent community's state of the art goes unexamined.

"Standard" means the first thing that community's referee asks, not an exotic one. Calibration by community type:

Engineering/applications: "does this survive the generic case (generic noise, generic inputs, adversarial settings), or only the structured case you studied?"
Experimentalists: "what does the apparatus actually measure, and is it the same object you computed? Which parts of the prediction survive the differences?"
Numericists: "is anything converged? In which units or scheme — and does the flattering choice hide the drift? What is the null hypothesis your signal must reject?"
Formal theorists: "is the named object well-defined here (does the symmetry, charge, or protection actually exist in this setting)? What is conjecture vs theorem, and does the abstract distinguish them?"
The incumbent method's community: "we already do this better — what is your edge, precisely, and have you checked our state of the art?"

对于受众表中的每一行，分别生成两项反对意见：

正确性：“这是否正确，并且在我所在社群关心的情况下是否成立？”
新颖性：“这是否有新意，相比我们已有的做法，具体进步究竟是什么？”

这是两份彼此不同的审稿报告，需要在不同位置回应：正确性应体现在范围声明、对照实验和比较中；新颖性应体现在引言的定位以及与既有工作的讨论中。文档只回答一个维度，就会死在另一个维度上；把两者混为一谈，会产生一种典型的混合体：文档证明了一切，却从未说明究竟有什么新意；或者宣称自己首创，却完全没有考察在位社群的最先进水平。

“标准”是指该社群的审稿人首先会问的问题，而不是某个罕见问题。应按社群类型校准：

工程/应用领域：“它能否在一般情形（一般噪声、一般输入、对抗性设置）下成立，还是只在你研究的结构化情形中成立？”
实验研究者：“实验装置究竟测量什么，是否与你计算的是同一对象？存在差异时，预测的哪些部分仍然成立？”
数值研究者：“是否确有收敛？采用什么单位或方案——讨巧的选择是否掩盖了漂移？信号必须拒绝的零假设是什么？”
形式理论研究者：“文中命名的对象在这里是否定义完善（对称性、电荷或保护是否确实在这一设定中存在）？哪些是猜想、哪些是定理，摘要是否将二者区分开？”
现有方法的社群：“我们早就把这件事做得更好——你的优势究竟是什么，是否检查过我们的最先进水平？”

3. Steelman every objection

3. 为每项反对意见做钢人化处理

The objection must be one a well-informed, unsympathetic expert would sign — steelmanned, never strawmanned. Three tests:

The sting test. If the document as written already answers the objection cleanly, you have probably written a weak one; sharpen until acting on it would require an edit. Occasionally the document really has pre-answered the strongest standard objection — treat that verdict as suspect and earn it, because a clean sweep is also exactly what a strawman pass produces.
The knowledge test. Write the objection in the referee's own voice and name what they know that the document ignores: the prior result, the standard control, the benchmark their community demands. If you cannot name what the objector knows, you have not simulated them; you have simulated yourself in their seat.
The lead test. A real referee leads with the standard objection. If your simulated objection is clever but nonstandard, generate the standard one first; the exotic one may follow as a second row, never as a substitute.

这项反对意见必须是充分知情且不会轻信的专家也会认可的异议——必须经过钢人化（steelman）处理，绝不能是稻草人式（strawman）质疑。需要通过三项测试：

刺痛测试（sting test）。如果按现有写法，文档已经能够干净利落地回答该异议，你很可能只写出了一个弱异议；应不断强化它，直到若要采纳这项异议就必须修改文档。偶尔，文档确实预先回答了最强的标准异议——此时应把这一裁决视为可疑，并通过实际生成过程来赢得这一结果，因为“全部通过”恰恰也可能是稻草人式审查产生的结果。
知识测试（knowledge test）。用审稿人自己的语气写下异议，并指出他们知道而文档忽略的内容：先前结果、标准对照实验、所在社群要求的基准测试。如果说不出反对者知道什么，你模拟的就不是他们，而是把自己放在了他们的位置上。
首要问题测试（lead test）。真正的审稿人一定会以标准反对意见开场。如果模拟出的异议很巧妙但并不标准，应先生成标准异议；罕见异议可以作为第二行记录，绝不能取代标准异议。

4. Verify each answer sits where the objector would look

4. 验证每项答案是否位于反对者会查找的位置

Hostile readers read in a fixed order: title, abstract, figures, conclusions — then, only if still engaged, the one section their objection lives in. "Derivable by assembling facts scattered across the document" is a FAIL: the test is whether the objector meets the answer on their actual reading path, at or before the point where the objection forms. A disqualifying caveat that first appears mid-body has already lost — the report was drafted at the abstract.

Severity: HIGH — unanswered objection a referee would lead with, or a claim the project's own data or files contradict. MEDIUM — answered in the body but absent where the impression forms. LOW — answered, could be more prominent.

怀有敌意的读者会按固定顺序阅读：标题、摘要、图表、结论——然后，只有仍有意继续阅读时，才会阅读其异议所在的那一节。“可由散布全文的事实拼合推导出来”视为 FAIL：测试标准是，反对者能否在实际阅读路径上、在异议形成之前或恰在其形成之处遇到答案。首次出现在正文中段的限定条件如果足以构成否决，就已经失败了——审稿报告是在阅读摘要时起草的。

严重程度：HIGH——审稿人会以此为首要异议提出，且尚未得到回答，或项目的自身数据或文件与该论断相矛盾。MEDIUM——正文已经回答，但答案未出现在形成整体印象的位置。LOW——已经回答，但还可以更加突出。

5. Audit the abstract separately and last

5. 单独审计摘要，并放在最后

The recurring failure: body honest, abstract clean of every caveat. The abstract must carry (a) any scope limitation a claimed audience would consider disqualifying if discovered later, and (b) conjecture-vs-established labeling for the headline claim. Audit it cold, as someone who will read nothing else — most referees form their frame there, and some readers are abstract-only.

反复出现的失败是：正文诚实，摘要却没有任何限定条件。摘要必须包含：(a) 所声称受众一旦稍后发现就会认为足以构成否决的任何适用范围限制；(b) 对核心论断标注其属于猜想还是已确立结论。应当脱离上下文地冷审摘要，就像一个只会阅读摘要的人那样——大多数审稿人的框架在那里形成，有些读者甚至只读摘要。

6. Apply minimal defensive edits only

6. 仅实施最小化防御性编辑

Scope sentences, null-hypothesis statements, prominence moves (caveat from body to abstract or scope paragraph), unit and convention consistency for headline numbers, conjecture labeling, terminology corrections. NO new claims, NO new results, and no hedge-blur: the correct response to an objection is one sentence stating the claim's boundary, never a softening of the claim everywhere it appears. Run all new text through the machine-voice/honesty linter; recompile or re-render and confirm clean.

包括范围界定句、零假设声明、显著性调整（将限定条件从正文移至摘要或范围段）、核心数字的单位与约定一致性、猜想标注以及术语修正。不得新增论断，不得新增结果，也不得进行模糊化对冲：对异议的正确回应是用一句话说明论断的边界，绝不是到处弱化论断。将所有新文本通过机器语气/诚信 linter；重新编译或重新渲染并确认结果无误。

7. Write the verdict table

7. 编写裁决表

referee_sim_<doc>.md

beside the document: audience / objection / axis (correctness or novelty) / answered-where-or-NOT / severity / fix applied. The table is the deliverable even when no edits are needed — it records that the audit ran and what it covered, and the next revision's audit starts from it.

在文档旁写入

referee_sim_<doc>.md

：受众／反对意见／维度（正确性或新颖性）／已回应位置或未回应／严重程度／已应用的修复。即使无需编辑，该表也是交付物——它记录了审查已经执行及其覆盖范围，下一版审查可从此开始。

Failure-mode catalogue

Failure-mode catalogue（失败模式目录）

The friendly reader. The simulated referee inherits the author's framing and raises only objections the document already answers; the pass returns clean, and the first genuine outsider leads with something the audit never generated. This is how the audit itself fails, and the reason for the sting test and the fresh-context rule.
The strawman objection. A cousin of the friendly reader: each objection is phrased just weakly enough that the existing text answers it, so the verdict table fills with reassuring rows and the audit certifies the very frame it was built to attack.
The unread answer. The objection is answered — thoroughly, honestly — in a subsection the objector never reaches. The referee formed the objection at the abstract and drafted the report before meeting the answer. Prominence is part of the answer, and an answer off the reading path is no answer.
Novelty–correctness conflation. One objection per audience instead of two. The document defends correctness exhaustively and never states its advance over the incumbent, or claims an advance while a prior result the incumbent community knows goes unengaged. Either way the report writes itself.
The missing audience. A community whose method, dataset, or benchmark the document borrows never gets a row — and it is exactly their standard objection that goes unanswered, because the borrowing invoked their standards without the audit noticing.
The abstract firewall. Every caveat lives in the body; the abstract is clean. Sentence-level honesty passes confirm each sentence individually and never see the assembly. Impressions form in the abstract; the body is the appeal, and most readers never hear the appeal.
The charitable outsider. The audit assumes a second field will read generously because the work is outside their specialty. The opposite holds: outsiders fall back on their home field's first-order concern and bring none of the insider's context for why a shortcut was reasonable.
Hedge-blur. The fix pass answers an objection by weakening the claim everywhere instead of scoping it once. The document now claims less than it established, reads as unconfident, and still leaves the objection unanswered.
Author-context contamination. The audit runs in the same context that wrote the document, so the "referees" judge each objection answered the way the author already did, blind spots intact. Distinct from the friendly reader: here even a well-generated objection receives a self-serving verdict on whether and where it was answered.

友好读者。 模拟审稿人继承作者的框架，只提出文档已经回答的异议；审查结果一片通过，而第一位真正的外部人士提出的却是审查从未生成的异议。这正是审查自身的失败方式，也是刺痛测试和新上下文规则存在的原因。
稻草人式反对意见。 这是友好读者的近亲：每项异议都被刻意写得足够弱，以致现有文本刚好能够回答；裁决表因此充满令人安心的记录，审查甚至为其本就旨在攻击的框架背书。
未被读到的答案。 异议在某个反对者永远不会读到的章节中得到了彻底而诚实的回答。审稿人在摘要处形成异议，并在遇到答案之前就已经起草了报告。显著性是答案的一部分，脱离阅读路径的答案不算答案。
新颖性—正确性混同。 每个受众只提出一项异议，而不是两项。文档详尽辩护正确性，却从未说明相较于在位方案有何进展；或者宣称有所进展，却没有讨论在位社群知晓的既有结果。无论哪一种，审稿报告都会自然写成。
遗漏的受众。 文档借用了某个社群的方法、数据集或基准测试，却没有为其设置审查行——因此 unanswered 的正是该社群的标准异议，因为借用他们的方法已经援引了他们的标准，而审查却没有意识到这一点。
摘要防火墙。 所有限定条件都留在正文中，摘要却毫无限定。句级诚信检查确认每个句子单独看来都是真实的，却看不到组合后产生的效果。整体印象在摘要中形成，正文才是申辩之处，而大多数读者根本听不到这项申辩。
善意的外部人士。 审查假定第二个领域会因为该工作不属于其专业范围而给予慷慨解读。事实恰恰相反：外部人士会退回到本领域的一阶关切，并且不会获得内部人士为捷径辩护时所需的背景信息。
模糊化对冲（hedge-blur）。 修复流程通过在所有位置弱化论断来回应异议。现在，文档声称的内容少于其实际证明的内容，读起来缺乏自信，而且仍然没有回答该异议。
作者上下文污染。 审查在与撰写文档相同的上下文中运行，因此这些“审稿人”会按照作者已有的回应方式判断每项异议是否得到回答，并让原有盲点继续存在。这与友好读者不同：在这里，即便异议生成得当，对其是否以及在哪里得到回应仍会得出自利裁决。

A worked example

A worked example（实例）

A document claims a faster method for a standard computation, demonstrated on a class of structured instances. The audience table, before any objection is written:

audience	why they get a row
incumbent method's community	the document claims to beat them
practitioners of the computation	they would act on the claim
numericists	the evidence is numerical
the structured-instance community	their instances serve as the benchmark

Steelmanned objections: the incumbent, on the novelty axis — "your comparison baseline is an implementation nobody uses; our current version handles your benchmark class in comparable time." The answer must live in the comparison section AND the abstract's claim must scope to what was actually beaten. Practitioners, on the correctness axis — "does this survive generic instances, or only the structured class you demonstrated?" The answer belongs in the scope paragraph and the abstract, since a practitioner discovering the restriction later would consider it disqualifying. Numericists — "converged in what sense, and against what null?" The answer belongs beside the headline figure, where the convergence impression forms. None of these objections is exotic; each is the first question its community asks. A version of this document that answers all three in those places is a materially safer document than the one that merely contains the answers somewhere.

一份文档声称一种更快的标准计算方法，并在一类结构化实例上进行了展示。在撰写任何异议之前，受众表如下：

受众	为何获得一行记录
在位方法的社群	文档声称击败了他们
该计算的使用者	他们会依据该论断采取行动
数值研究者	证据是数值形式的
结构化实例社群	他们的实例被用作基准测试

钢人化后的反对意见：在位方案在新颖性维度上提出——“你的比较基线是一项无人使用的实现；我们当前的版本处理你的基准类别时，耗时相当。”答案必须放在比较章节中，同时摘要中的论断必须限定在实际被击败的范围内。实践者在正确性维度上提出——“它能否在一般实例上成立，还是只在你展示的结构化类别中成立？”答案应放在范围段和摘要中，因为实践者若后来才发现这一限制，会认为它足以构成否决。数值研究者提出——“在何种意义上算是收敛，又是针对什么零假设？”答案应放在核心数字旁边，因为收敛印象正是在那里形成。这些异议都不新奇；每一个都是其社群首先会问的问题。一个在这些位置回答了全部三项异议的文档版本，在实质上会比仅仅在某处包含答案的版本更安全。

Cross-checks that pay off (run them every time)

Cross-checks that pay off（每次都值得执行的交叉检查）

Headline-number units: is the quoted number in the same units and convention the document's own equations define? (Failure shape: a drift quoted in a convenient intermediate scheme's units comes out at roughly half the value the paper's own equations define — the flattering choice hides a factor of two.)
Claims vs the project's own files: search the result notes for statements the document contradicts. (Failure shape: a document asserts a quantity must shrink with system size while the project's own larger-size results show it growing.)
Necessity language: every "must / guarantees / ensures" — is it derived, or is it hope written as necessity?
Null hypothesis: for any claimed signal (degeneracy, convergence to a special value, pattern), does the document state what chance would look like and why the data rejects it?
Temporal honesty: if an interpretation was found after the data forced it, the document may present theory-then-confirmation; restore the actual order. Independent prior results can still be cited as independent.
Conjecture labeling at the title and abstract level: "candidate / consistent with / designed for" vs language implying the claim is established.

核心数字的单位：引用的数字是否采用文档自身方程所定义的相同单位和约定？（典型失败形态：某个漂移量采用便于取值的中间方案单位表示时，数值约为论文自身方程所定义值的一半——讨巧的选择掩盖了一个二倍因子。）
论断与项目自身文件：搜索结果记录，找出与文档相矛盾的陈述。（典型失败形态：文档断言某个量必然随系统规模增大而减小，而项目自身在更大规模下的结果显示它在增大。）
必要性措辞：每一个“must／guarantees／ensures”是否有严格推导依据，还是把愿望写成了必然性？
零假设：对于任何声称存在的信号（简并、收敛到某个特殊值、模式），文档是否说明了偶然情况会是什么样子，以及数据为何排除了它？
时间顺序诚实性：如果某种解释是在数据迫使 researchers 得出的，文档却可能按照“先有理论、后得确认”的方式呈现；应恢复真实顺序。独立的先前结果仍可作为独立成果引用。
标题和摘要层面的猜想标注：“候选／与之相符／旨在用于”等措辞，与暗示论断已经确立的措辞是否明确区分？

Operational notes

Operational notes（操作说明）

Run the pass in a fresh agent context — never the context that authored the document. The simulated referees must not inherit the author's framing (see the catalogue's last entry).
The cost is one agent session per document. Against a referee report that leads with the objection you failed to simulate, it is cheap.
The exercise is a pre-mortem in Gary Klein's sense (Harvard Business Review, September 2007) run per audience.

请在全新的 Agent 上下文中运行本流程——绝不要使用撰写文档时的上下文。模拟的审稿人不得继承作者的框架（见失败模式目录的最后一项）。
每份文档的代价是一个 Agent 会话。相对于一份以你未能模拟的异议开头的审稿报告，这成本很低。
这项练习是加里·克莱因（Gary Klein）所定义意义下的事前验尸（pre-mortem），并针对每个受众分别执行（《哈佛商业评论》，2007 年 9 月）。

Expected yield (why this pass is mandatory)

Expected yield（预期收益）

First runs of this pass on finished, fact-verified documents reliably surface findings, often HIGH-severity ones: a generic-case scope statement missing from exactly the place an outside expert looks first; an abstract with no conjecture language over a fully-caveated body; a units choice that flatters a headline convergence figure; a claim the project's own files contradict. Each of these is a future referee-report finding, which is why the pass runs before circulation rather than after.

在已完成事实核验的文档上首次运行本流程时，通常都会可靠地暴露问题，其中往往包括 HIGH 严重程度的问题：恰好缺少一般情形的范围声明，而外部专家最先查看的正是那里；正文已处处带有完整限定，摘要却没有任何猜想措辞；一种单位选择美化了核心收敛数字；某项论断与项目自身文件相矛盾。每一种问题都会成为未来审稿报告中的发现项，这正是本流程要在传播之前而不是之后运行的原因。