independence-bookkeeping

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

independence-bookkeeping — keeping the second route honest

independence-bookkeeping — 保持第二条路径的诚实

THE LAW: independence is an auditable property of records, never an assumption. Two routes are independent when you can produce the trail showing what each one depends on — code, inputs, tuning history — and the trails do not meet at any layer where the feared error could live. If you cannot produce the trail, you do not have independence; you have a feeling, and correlated routes agree wrongly with exactly the same warm feeling as independent ones.
The strongest evidence a computed result can carry is agreement with a second route that could not have failed the same way. The entire content of that check is the "could not" — and the "could not" is a fact about records, not about intentions. Nobody sets out to build a circular check. Independence decays silently, through ordinary tidy-minded acts: a stale reference gets "refreshed" with the current code; two evaluators quietly converge on the same library; an oracle's values leak into the tuning of the thing the oracle later certifies. None of these announce themselves, and every one of them leaves the comparison script printing PASS. The only defense is bookkeeping kept from the start, because independence lost cannot be reconstructed afterward — once a value has touched a fit, no amount of later diligence untouches it.
法则:独立性是记录中可审计的属性,绝非假设。只有当你能产出轨迹, 说明每条路径依赖于什么——代码、输入、调优历史——并且两条轨迹在 任何可能藏有你所担心的错误的层级都不相交时,两条路径才算独立。 如果你无法产出这条轨迹,你拥有的就不是独立性,而只是一种感觉; 相关路径错误地达成一致时,带来的虚假安心感与独立路径完全相同。
计算结果所能携带的最强证据,是与一条不可能以同样方式失败的 第二条路径达成一致。该检查的全部内容就是“不可能”——而这个 “不可能”是关于记录的事实,而不是关于意图的事实。没有人会蓄意 构建循环检查。独立性会通过日常而看似严谨的举动悄然衰减: 陈旧参考值被当前代码“刷新”;两个 evaluator 悄悄收敛到同一个 library;oracle 的值泄漏进 oracle 随后所认证对象的调优过程。 这些情况无一会主动宣告自身,而每一个都会让比较脚本照样打印 PASS。 唯一的防线是从一开始就做好记录,因为失去的独立性事后无法重建—— 一个值一旦参与过 fit,之后无论多么谨慎,也无法消除这段影响。

The one-way contamination rule

单向污染规则

An oracle that fed a fit may never certify the result. Contamination is one-way and permanent. The moment a reference value influences any choice — a fitted parameter, a model form, a tolerance, a stopping decision, even which of several candidates you kept — that value belongs to the production side of the ledger forever. Certification draws only from the disjoint set: values the fitting process never saw and could not have seen.
This is stricter than it first looks. Influence includes the soft channels: you glanced at the reference while debugging and "fixed" the code until it agreed; you used it to decide how many terms were enough; you discarded a run because it disagreed with it. All of these are fits in everything but name. The test is counterfactual: if this reference had been different, could anything about the result have come out different? If yes, it fed the fit.
曾为某次 fit 提供输入的 oracle,绝不能认证该 fit 的结果。 污染是单向且永久的。参考值一旦影响任何选择——拟合参数、 模型形式、容差、停止决策,甚至你最终保留了多个候选中的哪一个—— 该值便永远属于账本的生产侧。认证只能从不相交的集合中取材: fit 过程从未见过、也不可能见过的值。
这比乍看之下更严格。影响也包括隐性渠道: 你在调试时瞥了一眼参考值,并“修补”代码直到两者一致; 你用它决定多少项才足够;又因为某次运行与其不一致而将其丢弃。 所有这些做法除了名字之外,实质上全都是 fit。检验方法是反事实: 如果该参考值不同,结果是否可能有任何不同? 若是,它就参与了 fit。

The ledger — what to write down and when

账本——记录什么,以及何时记录

Keep provenance as an append-only record, written at the moment of creation, not reconstructed at certification time. Memory of provenance is worthless; the whole point is that six weeks later nobody remembers which script produced the number.
1. A birth record for every reference value. Generator (program plus version or commit), inputs and settings, working precision, the estimated accuracy and where that estimate comes from (internal convergence, an error bound, agreement with something else — name it), and the date. A reference without a recorded accuracy cannot support a digit claim: a comparison is only as good as the weaker side, and an undocumented reference has no known strength. Comparisons have stalled at exactly the digit count of a reference nobody recorded — the agreement was real, and every claimed digit past it was manufactured.
2. A contact log for every oracle. Each time a value enters a fit, a tuning decision, a model selection, or a debugging session, append the event. The log is append-only; entries are never removed, because the rule above is one-way. Certification of claim C requires a reference whose contact log contains neither C nor anything C was built on.
3. A lineage audit for any claimed "two independent routes." Walk each route's dependency stack down to the libraries, the tabulated inputs, and the constants. Two programs in different languages that call the same special-function library share that library's defects; two derivations that read the same published table share its typos. Shared hardware and compilers rarely matter; shared mathematics and shared data almost always do. Write down the deepest shared layer and then argue, in writing, that the failure you are guarding against cannot live there. "Fully disjoint" is often impossible — that is fine; undocumented sharing is what kills you, not sharing itself.
4. One deliberately different engine. Keep an evaluator around precisely because it shares nothing with the rest of your stack — different algorithm, different language, different author if you can get one. It will be slower. Its value is not speed; its value is that its errors are uncorrelated with everyone else's, which makes its agreement worth more per digit than any in-family check.
5. The comparison itself is code — test it before believing it. Before any pass counts as evidence, feed the comparator a deliberately mismatched pair and watch it fail. A comparator that has never been seen to fail proves nothing: loose tolerances, a wrong column, a string comparison of truncated prints — all of these return agreement on everything, forever, and the output looks identical to a real pass. Also write down, before running the real comparison, what outcome would count as failure. A failure criterion chosen after seeing the numbers is a fit.
6. Close the loop in the record. When a comparison certifies a claim, the certificate cites the reference's birth record; the reference's contact log gains the claim. The reference is now burned for anything built on top of that claim. This is the price of using it, and the ledger is how you know the price was paid.
将溯源信息保留为仅可追加的记录,在创建当下写就, 而不是等到认证时再回溯重建。对溯源的记忆毫无价值; 关键就在于,六周后没人还记得哪个脚本产生了这个数字。
1. 每个参考值都有一份创建记录(birth record)。 生成器(程序及版本或 commit)、输入和设置、工作精度、估计准确度 及其依据(内部收敛、误差界限、与其他结果一致——必须写明), 以及日期。没有记录准确度的参考值无法支撑任何位数声明: 比较的效果只取决于较弱的一方,而没有文档的参考值强度未知。 比较往往恰好止于某个无人记录其准确度的参考值所对应的位数—— 一致性是真实的,而声称超出该位数的每一位都是制造出来的。
2. 每个 oracle 都有一份接触日志(contact log)。 每当一个值进入 fit、调优决策、模型选择或调试会话,都追加该事件。 日志仅可追加;条目永不删除,因为上述规则是单向的。 认证主张 C 需要一个参考值,其接触日志既不包含 C, 也不包含构建 C 所依据的任何内容。
3. 对任何所谓“两条独立路径”的主张,都要做谱系审计(lineage audit)。 沿每条路径的依赖栈一直下钻到 library、表格化的输入和常量。 用不同语言编写、却调用同一个 special-function library 的两个程序, 会共享该 library 的缺陷;读取同一张已发布表格的两种推导, 会共享其中的错漏。共享 hardware 和 compilers 很少会造成影响; 共享数学和共享 data 几乎总会造成影响。写下最深的共享层, 再以书面论证你正要防范的故障不可能存在于该层。 “完全不相交”通常不可能——这没关系; 真正致命的是未记录的共享,而不是共享本身。
4. 刻意保留一个不同的 engine。 之所以保留某个 evaluator,正是因为它与其余技术栈没有任何共享—— 不同算法、不同语言;如果能找到不同作者,则更好。 它会更慢。它的价值不在速度; 而在于它的错误与其他评估器的错误都不相关, 这使它每提供一位数字的一致性,都比任何同族检查更有价值。
5. 比较本身就是代码——先测试,再相信它。 在任何 PASS 能算作证据之前,给 comparator 输入一对故意不匹配的值, 并观察它失败。一个从未被看到失败的 comparator 什么也证明不了: 过宽的容差、错误的列、对截断后的打印结果做字符串比较—— 这些做法都会永远对任何内容返回一致, 而且输出看起来与真正的 PASS 完全相同。 你还必须在运行真实比较之前写明,什么结果算作失败。 看到数字之后才选定的失败标准,本身就是一次 fit。
6. 在记录中闭合回路。 当比较认证某项主张时,证书必须引用该参考值的创建记录; 参考值的接触日志中要新增这项主张。 此后,凡是建立在这项主张之上的内容,都不得再使用该参考值—— 它已被“烧毁”。这是使用它的代价,而账本让你知道这项代价确已付出。

Failure-mode catalogue

失败模式目录

The classes this discipline exists to catch. Every one of them prints PASS.
  • The shared-library twins. Two "independent" implementations both call the same underlying expansion; a defect there makes them agree, wrongly, to full precision — the check measures the library's self-consistency.
  • The regenerated reference. A stored reference goes stale or missing and someone helpfully re-derives it with the current code; from that day the check compares the program with itself and can never fail again.
  • Circular calibration. Route A is tuned until it matches route B; route B is later "validated" by its agreement with route A. Each certificate cites the other; nothing external anchors either one.
  • The leaky oracle. Reference values were used to choose among candidate models — keep the form that matches — and then the same references certify the chosen model; the certificate measures the selection, not the truth.
  • The undocumented-precision stall. The comparison "confirms to many digits", but the reference was only ever good to fewer, a fact recorded nowhere; every digit past the reference's true accuracy is fiction.
  • Contamination one step removed. The raw values are properly held out, but the "independent" check compares residuals computed with the fitted parameters — the held-out quantity is a function of the fit.
  • The comparator that cannot fail. Tolerance too loose, wrong field compared, truncated strings matched — the harness agrees with anything, and nobody ever planted a known mismatch to find out.
  • The helpful cache. A shared cache or memoization layer serves route B the value route A stored; two routes return the same number because it is the same number.
  • The convenient partial audit. Independence declared after inspecting the top layer only; three layers down, both routes read the same input table.
  • Post-hoc promotion. A value computed as a quick internal cross-check is promoted months later into the official certificate, its casual birth record never upgraded to match its new weight.
本规程用于捕捉的失败类别。每一类都会打印 PASS。
  • 共享 library 的孪生实现。 两个“独立”实现都调用同一个底层 expansion; 该处存在缺陷时,它们会错误地达到全精度一致—— 这项检查测量的只是 library 的自洽性。
  • 被重新生成的参考值。 已存储的参考值过期或丢失, 而有人“好心地”用当前代码重新推导;从那天起, 检查就是程序与自身比较,再也不可能失败。
  • 循环校准。 路径 A 被调优到与路径 B 一致; 之后路径 B 又因其与路径 A 一致而被“验证”。 每份证书都引用另一份;二者都没有任何外部锚点。
  • 泄漏的 oracle。 参考值曾被用于在候选模型间做选择—— 保留与之一致的形式——随后又用同一批参考值认证所选模型; 证书衡量的是选择过程,而不是真实性。
  • 未记录精度导致的停滞。 比较“确认到很多位”, 但参考值从来只精确到更少的位数,而这一事实没有记录在任何地方; 超出参考值真实准确度的每一位都是虚构。
  • 隔了一步的污染。 原始值被正确留出, 但所谓“独立”检查比较的是使用拟合参数计算的残差—— 留出的量是 fit 的函数。
  • 不可能失败的 comparator。 容差过松、比较了错误的字段、 匹配了截断字符串——测试 harness 会同意任何结果, 而且从来没人植入已知不匹配样本来查明这一点。
  • 好心的 cache。 共享 cache 或 memoization 层 把路径 A 存储的值提供给路径 B; 两条路径返回同一个数字,因为它们用的本来就是同一个数字。
  • 方便但局部的 audit。 只检查最上层就宣布独立性; 再往下三层处,两条路径读取的是同一张输入表。
  • 事后晋升。 某个最初仅作为快速内部交叉检查而计算出的值, 几个月后被提升为正式认证依据; 其随手创建的 birth record 从未按其新增分量加以升级。

Worked micro-example: certifying a fitted constant

实例演示:认证一个拟合常数

You fit a linear combination of candidate constants to a high-precision value and the fit closes beautifully. What certifies it?
Not agreement with the value you fit to — that is residual arithmetic restated, evidence of nothing. Certification needs digits the fit never saw: extend the target value past the precision used in the fit, using an evaluator that did not produce the fit input, and check the fitted form against the extension; or evaluate both sides at a different point where an untouched reference exists. Add the destructive control: refit with one supposedly essential candidate removed and confirm the fit collapses. A fit that survives the removal of its key ingredient was never measuring that ingredient.
A birth record small enough that there is no excuse to skip it:
value:     target integral at the check point
accuracy:  60 digits (internal convergence estimate; last 5 digits unconfirmed)
generator: series evaluator v3.2 (commit <hash>), 80-digit working precision
date:      <date written at creation>
contacts:  fed the ansatz fit of <date>  -> may not certify that fit
Five lines. The last one is the ledger doing its job: whoever reaches for this value at certification time is told, by the record and not by memory, that it is burned.
你将候选常数的线性组合拟合到一个高精度值,且 fit 收敛得非常漂亮。 什么可以认证它?
不能用你拟合所依据的数值来证明一致—— 那只是把残差运算换个说法重述一遍,毫无证据。 认证需要 fit 从未见过的位数:使用一个未生成 fit 输入的 evaluator, 将目标值延展到高于 fit 所用精度的精度, 再检查拟合后的形式与扩展值; 或者在另一个存在未受污染参考值的位置上计算两侧。 加入破坏性对照:移除一个据称必不可少的候选项后重新 fit, 并确认 fit 崩溃。若某次 fit 在移除其关键成分后仍然成立, 它原本就从未测量该成分。
一份小到让人没有理由跳过的创建记录:
value:     target integral at the check point
accuracy:  60 digits (internal convergence estimate; last 5 digits unconfirmed)
generator: series evaluator v3.2 (commit <hash>), 80-digit working precision
date:      <date written at creation>
contacts:  fed the ansatz fit of <date>  -> may not certify that fit
只有五行。最后一行正是在发挥账本的作用: 任何人在认证时打算取用这个值,记录而非记忆都会告诉他, 这个值已经被“烧毁”。

Closing checklist

收尾检查清单

Before calling any agreement a certification:
  • Every reference has a birth record: generator + version, inputs, accuracy with its basis, date — written at creation, not recalled.
  • Every oracle has a contact log; every fit, tuning, selection, and debugging use is in it.
  • The certifying reference is disjoint from the claim: not in its contact history, not produced by the code under test.
  • The counterfactual test passes: had the reference differed, the result could not have come out different.
  • For "two independent routes": the deepest shared layer is named in writing, with the argument that the feared error cannot live there.
  • The comparator has been shown to FAIL on a planted mismatch.
  • The failure criterion was written down before the comparison ran.
  • Claimed agreement digits do not exceed the recorded accuracy of the weaker side.
  • The certificate cites the birth record; the contact log now records the certification, burning the reference for anything built on it.
If any box is unchecked, the honest statement is "consistent, pending an independent check" — and the word independent stays out of the writeup until the ledger can back it.
The empirical basis for distrusting "independent" implementations that share a layer is Knight and Leveson (IEEE Trans. Softw. Eng. SE-12 (1986) 96); the one-way contamination rule is the train/test separation of statistical learning stated for oracles.
在把任何一致性称为认证之前:
  • 每个参考值都有创建记录:生成器及版本、输入、 附有依据的准确度、日期——在创建时写就,而非事后回忆。
  • 每个 oracle 都有接触日志; 每一次 fit、调优、选择和调试使用都记录在内。
  • 用于认证的参考值与主张不相交: 该主张未出现在参考值的接触历史中, 参考值也并非由受测代码生成。
  • 反事实检验通过:如果参考值不同, 结果不可能有任何不同。
  • 对于“两条独立路径”,已书面标明最深的共享层, 并论证所防范的故障不可能存在于该层。
  • 已证明 comparator 会在植入的不匹配情况下输出 FAIL。
  • 失败标准在比较运行前已写明。
  • 所声称的一致位数不超出较弱一方记录的准确度。
  • 证书引用创建记录;接触日志现已记录本次认证, 从而烧毁该参考值,禁止将其用于任何建立在该主张之上的内容。
只要有任何一项未勾选,诚实的表述就是 “结果一致,尚待独立检查”——在账本能够提供依据之前, 报告中不要使用独立一词。
质疑那些共享某一层的所谓“独立”实现,其经验依据来自 Knight and Leveson (IEEE Trans. Softw. Eng. SE-12 (1986) 96); 单向污染规则则是把统计学习中的训练/测试分离原则 表述为对 oracle 的规则。