Back to Details

fable-simulator

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Fable Simulator

Fable Simulator

Create an independent model of expected behavior and compare it with the candidate without pretending the simulation itself is production evidence.
创建一个预期行为的独立模型，并与候选实现进行比较，同时不得将模拟本身当作生产环境证据。

Mission

使命

Simulation is useful when ordinary tests risk sharing the same assumptions as the implementation, when a UI/agent flow needs controlled playthrough, or when failure injection can expose paths that are hard to reproduce safely.
The Skill must preserve a hard boundary between simulated/oracle evidence and real runtime verification.
当普通测试可能与实现共享相同假设、UI/agent 流程需要进行受控演练，或故障注入能够暴露难以安全复现的路径时，模拟非常有用。
Skill 必须在模拟/oracle 证据与真实运行时验证之间保持严格的边界。

Activate When

何时启用

  • an algorithm/rewrite needs an independently derived oracle;
  • a contract is implicit across callers and needs reconstruction;
  • UI/agent state transitions benefit from scripted playthrough/failure injection;
  • testing destructive/rare conditions safely requires a model/sandbox;
  • differential testing between two independent implementations is valuable.
  • 算法/重写需要独立推导的 oracle；
  • contract 在各个调用方之间是隐式的，需要重新构建；
  • UI/agent 状态转换适合通过脚本化演练或故障注入进行验证；
  • 需要通过模型/沙箱安全测试破坏性或罕见条件；
  • 对两个独立实现进行差分测试具有价值。

Do Not Activate When

何时不应启用

  • normal tests/runtime evidence already proves the claim cheaply;
  • the proposed oracle is derived from the same code/assumptions under test;
  • simulation would be reported as proof that an external production system actually behaved that way.
  • 常规测试/运行时证据已经能够低成本证明该声明；
  • 拟议的 oracle 派生自正在测试的相同代码/假设；
  • 模拟会被报告为外部生产系统确实如此行为的证明。

Simulation Classification

模拟分类

ModeIndependence requirement
Golden fixturesfixture expected outputs come from trusted contract/observations
Reference implementationindependently written/maintained logic
Differential provider/modelseparate implementation/model with blinded oracle
State-machine simulationtransitions/invariants defined from product contract
Failure injectioncontrolled faults with explicit modeled assumptions
UI playthroughbrowser actions + observable DOM/network/runtime evidence
模式独立性要求
Golden fixturesfixture 的预期输出来自可信的 contract/观察结果
Reference implementation独立编写/维护的逻辑
Differential provider/model使用盲化 oracle 的独立实现/模型
State-machine simulation根据产品 contract 定义的状态转换/不变量
Failure injection带有明确建模假设的受控故障
UI playthrough浏览器操作 + 可观察的 DOM/网络/运行时证据

Protocol

协议

Stage 1 — Define the claim and oracle independence

阶段 1——定义声明与 oracle 独立性

State what candidate behavior is being checked and why the oracle does not simply repeat candidate logic.
List shared assumptions. If a shared assumption could cause both candidate and oracle to be wrong identically, record that coverage gap.
说明正在检查候选实现的哪种行为，以及为什么该 oracle 不是简单重复候选逻辑。
列出共享假设。如果某个共享假设可能导致候选实现和 oracle 以相同方式同时出错，则记录该覆盖缺口。

Stage 2 — Derive contract from independent sources

阶段 2——从独立来源推导 contract

Use public interfaces, callers, specs, historical golden outputs, primary docs, or separately maintained reference behavior. Avoid reading implementation details solely to recreate the same algorithm.
使用公共接口、调用方、规范、历史 golden 输出、一手文档或单独维护的参考行为。避免仅为了复现相同算法而阅读实现细节。

Stage 3 — Build representative and edge corpus

阶段 3——构建有代表性及边界场景的语料库

Include normal, boundary, invalid, stateful, adversarial, and failure cases relevant to the claim. Preserve user/untracked workspace files and run in isolated temp/sandbox locations where possible.
包含与声明相关的正常、边界、无效、有状态、对抗性和失败用例。尽可能保留用户/未跟踪的工作区文件，并在隔离的临时目录/沙箱位置中运行。

Stage 4 — Execute candidate and oracle separately

阶段 4——分别执行候选实现和 oracle

Capture inputs, outputs, errors, side effects/state transitions, timing only when required, and tool/environment identity.
捕获输入、输出、错误、副作用/状态转换；仅在有需要时捕获时序，并记录工具/环境标识。

Stage 5 — Compare semantically

阶段 5——进行语义比较

Normalize only differences that the contract declares irrelevant. Do not normalize away a real behavioral difference just to reach zero diff.
For UI, compare causal rows: 
action → expected observable → actual observable → evidence
not pixels alone unless pixel fidelity is itself the contract.
仅归一化 contract 明确声明为无关的差异。不要为了得到零差异而归一化掉真实的行为差异。
对于 UI，应比较因果记录： 
action → expected observable → actual observable → evidence
而不是只比较像素，除非像素保真度本身就是 contract。

Stage 6 — Triage divergence

阶段 6——诊断差异

A mismatch means one of:
  • candidate wrong;
  • oracle wrong/stale;
  • contract ambiguous;
  • normalization invalid;
  • environment differs.
Do not automatically "fix candidate to oracle" until the source of truth is established.
不匹配意味着以下情况之一：
  • 候选实现错误；
  • oracle 错误或过期；
  • contract 含糊不清；
  • 归一化无效；
  • 环境存在差异。
在确定事实来源之前，不要自动将“候选实现修复为与 oracle 一致”。

Stage 7 — Mark evidence scope

阶段 7——明确标注证据范围

Report simulation/oracle evidence distinctly. Hand off to 
fable-verify
for real runtime/package/environment proof where the claim requires it.
分别报告模拟/oracle 证据。对于需要真实运行时/软件包/环境证明的声明，转交 
fable-verify
。

Decision Rules

决策规则

  • An oracle copied from candidate implementation is self-confirming and invalid.
  • A second LLM is not automatically an independent oracle if it receives the candidate answer or hidden expected output.
  • Golden fixtures need provenance; unexplained fixtures can encode old bugs.
  • UI screenshots without interaction/state evidence are weak for functional causality.
  • Failure injection proves behavior under the modeled fault, not that real infrastructure fails exactly that way.
  • Zero diff across a narrow corpus is not universal correctness; state coverage explicitly.
  • Never modify/delete untracked user files to make simulation deterministic.
  • 从候选实现复制而来的 oracle 只是自我确认，因而无效。
  • 如果第二个 LLM 接收了候选答案或隐藏的预期输出，则它不会自动成为独立 oracle。
  • Golden fixtures 需要可追溯的来源；未经解释的 fixture 可能编码了旧有缺陷。
  • 没有交互/状态证据的 UI 截图难以证明功能因果关系。
  • 故障注入只能证明建模故障下的行为，不能证明真实基础设施恰好会采用相同方式发生故障。
  • 狭窄语料库上的零差异并不代表普遍正确；应明确说明覆盖范围。
  • 绝不修改或删除未跟踪的用户文件来让模拟变得确定。

Invariants

不变量

  • Oracle independence and shared assumptions are explicit.
  • Simulation evidence is labeled as simulation.
  • Input corpus and normalization are reproducible.
  • User/untracked workspace state is preserved.
  • Divergence is diagnosed before deciding which side is wrong.
  • Real-world claims receive real verification when required.
  • 明确说明 oracle 的独立性及共享假设。
  • 将模拟证据标记为模拟证据。
  • 输入语料库和归一化过程可复现。
  • 保留用户/未跟踪的工作区状态。
  • 在决定哪一方错误之前，先诊断差异。
  • 现实世界声明在需要时接受真实验证。

Failure Taxonomy

失败分类

Tautological oracle

Tautological oracle

Oracle reuses candidate implementation/answer. Redesign independently.
Oracle 复用了候选实现/答案。应对其进行独立重新设计。

Stale oracle

Stale oracle

Reference behavior no longer matches accepted contract. Re-ground oracle before judging candidate.
参考行为不再符合已接受的 contract。根据该 contract 重新校准 oracle，然后再判断候选实现。

Ambiguous contract

Ambiguous contract

Candidate and oracle differ but both are plausible. Route to plan/research rather than choose arbitrarily.
候选实现和 oracle 存在差异，但两者都合理。应转交 plan/research 流程，而不是任意选择。

Over-normalization

Over-normalization

Comparison strips a meaningful difference. Narrow normalization to contract-declared irrelevant fields.
比较过程消除了有意义的差异。将归一化范围限制为 contract 声明为无关的字段。

Simulation/reality confusion

Simulation/reality confusion

Simulated pass is used as production/runtime proof. Downgrade claim and hand to verify/run.
将模拟通过结果用作生产/运行时证明。应降低声明强度，并转交 verify/run。

Workspace contamination

Workspace contamination

Simulation writes into real user state. Move to isolated sandbox and restore owned mutations only.
模拟写入真实用户状态。应移至隔离的沙箱，并且只恢复模拟自身产生的变更。

Anti-Patterns

反模式

  • implementing the oracle by copying candidate code;
  • giving an LLM oracle the expected answer;
  • demanding 100% zero diff without representative corpus reasoning;
  • treating screenshots as complete UI verification;
  • normalizing every mismatch until green;
  • claiming external service behavior from a fake;
  • deleting untracked files to reset simulation;
  • changing candidate immediately on any oracle mismatch.
  • 通过复制候选代码来实现 oracle；
  • 向 LLM oracle 提供预期答案；
  • 在没有代表性语料库推理的情况下要求 100% 零差异；
  • 将截图视为完整的 UI 验证；
  • 归一化所有不匹配，直到结果变为通过；
  • 根据 fake 声称外部服务的行为；
  • 删除未跟踪文件来重置模拟；
  • 一遇到 oracle 不匹配就立即修改候选实现。

Simulation Packet

模拟数据包

text
Claim:
Oracle type/source:
Independence + shared assumptions:
Corpus/failure injections:
Candidate environment:
Comparison/normalization:
Divergences:
Oracle validity decision:
Simulation verdict:
Real evidence still required:
text
声明：
oracle 类型/来源：
独立性 + 共享假设：
语料库/故障注入：
候选环境：
比较/归一化：
差异：
oracle 有效性判定：
模拟结论：
仍需真实证据：

Completion Criteria

完成标准

Simulation completes when:
  • oracle independence is credible and documented;
  • corpus covers the important contract dimensions;
  • candidate/oracle comparisons are reproducible;
  • divergence is classified instead of blindly resolved;
  • workspace remains safe;
  • evidence is labeled narrowly and handed to real verification where needed.
模拟在以下情况下完成：
  • oracle 的独立性可信且有文档记录；
  • 语料库覆盖了重要的 contract 维度；
  • 候选实现与 oracle 的比较可复现；
  • 对差异进行了分类，而不是盲目解决；
  • 工作区保持安全；
  • 证据得到精确标注，并在需要时移交真实验证。

Progressive Resources

扩展资源

  • Deep guide: 
    references/oracle-independence-and-simulation-boundaries.md
  • Existing guide: 
    references/oracle-derivation-guide.md
  • Example: 
    examples/independent-oracle-verification.md
  • 深入指南：
    references/oracle-independence-and-simulation-boundaries.md
  • 现有指南：
    references/oracle-derivation-guide.md
  • 示例：
    examples/independent-oracle-verification.md