Back to Details

failure-flywheel

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Skill: failure-flywheel

Skill: failure-flywheel

Turn recurring failures into rules. Capture by code, count deterministically, draft a stub for a human to approve — never auto-promote.
将重复出现的失败转化为 rules。按 code 捕获失败、确定性地计数，并为人工审批起草 stub——绝不自动 promote。

WHAT

是什么

Purpose và context

目的与上下文

  • Purpose: biến lỗi lặp lại của agent thành guardrail: ghi lỗi bằng code, đếm tất định theo taxonomy, khi một lớp lỗi vượt ngưỡng thì dựng STUB đề xuất rule/skill vào luồng propose→gate.
  • Trigger (when to use):
    • A task failed in a way that could recur (invented an API, broke a test, ignored a rule, over-built). Record it so it is counted, not forgotten.
    • Periodically (end of session, or every N failures): 
      --report
      to see which failure class recurs most.
    • When a class crosses the recurrence threshold: 
      --draft <category>
      to seed a candidate rule, then run 
      /propose
      to finish it.
    • User nói "log this failure", "record the mistake", "why does this keep happening", "turn failures into rules", "failure taxonomy", "what keeps breaking", hoặc 
      /failure-flywheel
      .
  • Non-goals: Do NOT use to auto-write rules. This skill seeds the gate; a human always approves. Không tự chưng cất nội dung rule (10% cần phán đoán thuộc về người + 
    /propose
    ).
  • 目的： 将 Agent 的重复失败转化为 guardrail：按 code 记录失败，按 taxonomy 确定性地计数；当某个失败类别超过阈值时，在 propose→gate 流程中生成 rule/skill 候选 STUB。
  • 触发条件（何时使用）：
    • 任务以可能重复的方式失败（虚构了 API、破坏了测试、无视了 rule、过度构建）。记录该失败，使其被计数而不是被遗忘。
    • 定期（会话结束时，或每 N 次失败后）：运行 
      --report
      查看最常重复的失败类别。
    • 当某个类别越过重复阈值时：运行 
      --draft <category>
      播种候选 rule，然后运行 
      /propose
      完成它。
    • 用户说出“log this failure”“record the mistake”“why does this keep happening”“turn failures into rules”“failure taxonomy”“what keeps breaking”，或 
      /failure-flywheel
      。
  • 非目标： 不要用于自动编写 rules。此 skill 只为 gate 播种；始终由人工审批。不自动提炼 rule 内容（10% 需要人的判断，并配合 
    /propose
    ）。

Mental model

思维模型

failure → record(category) → failures.jsonl → --report (group · COUNT · leaderboard) → lớp ≥ recurrence_threshold → --draft stub → /propose → cổng người duyệt
. Hamel: create → label → fix → repeat. Máy làm 90% cơ học; 10% "rule thực sự là gì" để người.
failure → record(category) → failures.jsonl → --report (group · COUNT · leaderboard) → 类别 ≥ recurrence_threshold → --draft stub → /propose → 人工 gate
。Hamel：create → label → fix → repeat。机器完成 90% 的机械工作；剩下 10% 的“rule 究竟是什么”交给人判断。

Input và output contract

输入与输出约定

FieldRequired?Ý nghĩa
In
<category>
+ 
"<summary>"
có (record)lớp taxonomy gần nhất; category lạ vẫn ghi, bị gắn cờ ở report
In
--detail "..."
khôngchi tiết thêm
In
--date YYYY-MM-DD
không (draft)ISO only, mặc định 2026-06-29; sai định dạng → exit 2
In
harness/failure-flywheel.config.yaml
cóngưỡng + taxonomy (adapter, 
verified: false
)
Outdòng trong failures.jsonlcó (record)lịch sử local, gitignored
Outleaderboardcó (report)đếm theo lớp, first/last seen, lớp nào 
DRAFT
Out
llmwiki/wiki/sources/draft/DDMMYY-failure-<category>.md
chỉ khi đủ ngưỡngSTUB (OKF frontmatter + 
## Origin
, không 
## Plan
) — dừng chờ người
字段必需？含义
输入
<category>
+ 
"<summary>"
是（record）最接近的 taxonomy 类别；即使 category 未知也仍会记录，并在 report 中标记
输入
--detail "..."
否补充详情
输入
--date YYYY-MM-DD
否（draft）仅限 ISO，默认 2026-06-29；格式错误 → exit 2
输入
harness/failure-flywheel.config.yaml
是阈值 + taxonomy（adapter，
verified: false
）
输出failures.jsonl 中的一行是（record）本地历史，gitignored
输出leaderboard是（report）按类别计数、首次/最近出现时间，以及哪些类别为 
DRAFT
输出
llmwiki/wiki/sources/draft/DDMMYY-failure-<category>.md
仅在达到阈值时STUB（OKF frontmatter + 
## Origin
，无 
## Plan
）——等待人工审批

Rules và capabilities

规则与能力

  • RULE-01 (MUST): Capture is fail-open — recording a failure must never break the session.
  • RULE-02 (MUST): Never auto-promote. 
    --draft
    seeds 
    /propose
    ; a human approves.
  • RULE-03 (MUST): The draft filename date is fixed (
    --date
    , default 2026-06-29, ISO 
    YYYY-MM-DD
    only     — not the 
    DDMMYY
    wiki filename convention) for determinism; real use can stamp today. A malformed 
    --date
    now exits 2 with a clear error instead of silently falling back to the default (fixed 2026-07-20 after that exact silent-fallback shipped a wrong-dated draft file).
  • RULE-04 (MUST): Do not hard-code the threshold or taxonomy anywhere but the config — that is the adapter.
  • Capabilities: ghi append lịch sử lỗi local; đọc config ngưỡng/taxonomy; ghi một file draft stub vào wiki draft. Không mạng, không sửa rule/skill thật.
  • RULE-01 (MUST)：捕获必须采用 fail-open——记录失败绝不能中断 session。
  • RULE-02 (MUST)：绝不自动 promote。
    --draft
    只为 
    /propose
    播种；必须由人工审批。
  • RULE-03 (MUST)：draft 文件名中的日期由 
    --date
    确定（默认 2026-06-29，仅限 ISO 
    YYYY-MM-DD
    ——不采用 
    DDMMYY
    wiki 文件名约定），以确保确定性；实际使用时可以标记当天。格式错误的 
    --date
    现在会 exit 2 并给出清晰错误，而不是静默回退到默认值。该静默回退曾生成日期错误的 draft 文件；现已修复（2026-07-20）。
  • RULE-04 (MUST)：除 config（adapter）外，不得在任何位置硬编码 threshold 或 taxonomy。
  • 能力：追加写入本地失败历史；读取 config 中的 threshold/taxonomy；向 wiki draft 写入一个 draft stub。不联网，不修改真实的 rule/skill。

Failure boundaries

失败边界

  • Ghi lỗi hỏng → fail-open (session không vỡ), không chặn.
  • --draft
    lớp chưa đủ ngưỡng → không viết stub (blocked hẹp, báo đếm hiện tại).
  • --date
    sai định dạng → failed exit 2, thông báo rõ.
  • Config còn 
    verified: false
    → stub chỉ có human-TODO, không rule tự viết (partial có chủ đích).
  • 记录失败时发生错误 → fail-open（session 不会崩溃），不阻塞。
  • --draft
    对未达到阈值的类别 → 不写入 stub（仅阻断此项操作），并报告当前计数。
  • --date
    格式错误 → failed exit 2，并给出明确消息。
  • Config 仍为 
    verified: false
    → stub 只包含 human-TODO，不自动编写 rule（有意保持 partial）。

HOW

如何执行

Main workflow

主流程

StepTypeInputsActionOutputs/exitFailure/next
W01effectfailure vừa xảy ra
record <category> "<summary>"
dưới lớp taxonomy gần nhất		dòng jsonlcategory lạ → vẫn ghi + cờ ở report
W02deterministicfailures.jsonl + config
--report
đọc leaderboard		lớp đánh dấu 
DRAFT
nếu ≥ ngưỡng		không lớp nào đủ → dừng
W03effectlớp đủ ngưỡng
--draft <category>
dựng stub		file draft stubchưa đủ ngưỡng / 
--date
sai → dừng
W04judgmentstubSTOP chờ người; người chạy 
/propose
để thành rule/skill có gate		proposal đầy đủngười từ chối → giữ stub
Chi tiết từng bước (nguồn chân lý cho W01–W04):
步骤类型输入操作输出/退出码失败/下一步
W01effect刚发生的失败在最接近的 taxonomy 类别下运行 
record <category> "<summary>"
jsonl 行category 未知 → 仍记录，并在 report 中标记
W02deterministicfailures.jsonl + config运行 
--report
读取 leaderboard		类别 ≥ 阈值时标记为 
DRAFT
无类别达到阈值 → 停止
W03effect达到阈值的类别运行 
--draft <category>
生成 stub		draft stub 文件未达到阈值 / 
--date
错误 → 停止
W04judgmentstubSTOP，等待人工；由人运行 
/propose
，生成带 gate 的完整 rule/skill		完整 proposal人拒绝 → 保留 stub
各步骤详情（W01–W04 的事实来源）：

The flywheel (Hamel: create -> label -> fix -> repeat)

飞轮（Hamel: create -> label -> fix -> repeat）

The slow human step is "notice the same mistake keeps happening and turn it into a rule." This makes the mechanical 90% deterministic (capture + taxonomy + count + scaffold) and refuses to fake the 10% that needs judgement (what the rule actually is).
缓慢的人工步骤是“发现同一个错误反复发生，并将其转化为 rule”。这样，90% 的机械工作（捕获 + taxonomy + 计数 + scaffold）都是确定性的，同时不会假装替代那 10% 需要判断的工作（rule 究竟是什么）。

Commands

命令

bash
undefined
bash
undefined

1. CAPTURE — append one failure (BY CODE; failures.jsonl is gitignored local history)

1. CAPTURE — append one failure (BY CODE; failures.jsonl is gitignored local history)

python3 harness/scripts/failure-flywheel.py record <category> "<summary>" [--detail "..."]
python3 harness/scripts/failure-flywheel.py record <category> "<summary>" [--detail "..."]

2. REPORT — deterministic taxonomy: group by category, COUNT, leaderboard (most-frequent

2. REPORT — deterministic taxonomy: group by category, COUNT, leaderboard (most-frequent

first) + first/last seen + which classes are eligible to draft

first) + first/last seen + which classes are eligible to draft

python3 harness/scripts/failure-flywheel.py --report
python3 harness/scripts/failure-flywheel.py --report

3. DRAFT — only if <category> recurs >= recurrence_threshold: scaffold a STUB proposal at

3. DRAFT — only if <category> recurs >= recurrence_threshold: scaffold a STUB proposal at

llmwiki/wiki/sources/draft/DDMMYY-failure-<category>.md and STOP for human approval

llmwiki/wiki/sources/draft/DDMMYY-failure-<category>.md and STOP for human approval

python3 harness/scripts/failure-flywheel.py --draft <category> [--date YYYY-MM-DD]
undefined
python3 harness/scripts/failure-flywheel.py --draft <category> [--date YYYY-MM-DD]
undefined

Steps

步骤

  1. On a failure, 
    record
    it under the closest taxonomy category (see the config). Capture never blocks — an unlisted category is still recorded and flagged in 
    --report
    .
  2. Run 
    --report
    to read the leaderboard. A class marked 
    DRAFT
    has recurred >= the threshold.
  3. For an eligible class, 
    --draft <category>
    . It writes a valid draft STUB (OKF frontmatter + 
    ## Origin
    , deliberately no 
    ## Plan
    so it passes the R7 gate as a seed) containing a templated "TODO: distill rule from these N failures" + the failure list.
  4. The stub STOPS for you. Run 
    /propose
    to turn it into a complete, gated rule/skill. FailureFlywheel never auto-promotes — the gate stays human.
  1. 发生失败时，在最接近的 taxonomy 类别下运行 
    record
    。捕获绝不能阻塞——未列出的 category 仍会被记录，并在 
    --report
    中标记。
  2. 运行 
    --report
    读取 leaderboard。标记为 
    DRAFT
    的类别表示其重复次数已 ≥ 阈值。
  3. 对于符合条件的类别，运行 
    --draft <category>
    。它会写入一个有效的 draft STUB（OKF frontmatter + 
    ## Origin
    ，有意不含 
    ## Plan
    ，从而以种子的形式通过 R7 gate），其中包含模板化的 
    TODO: distill rule from these N failures
    及失败列表。
  4. stub 在此 STOP，等待你接手。运行 
    /propose
    ，将其转化为完整且带 gate 的 rule/skill。FailureFlywheel 绝不自动 promote——始终由人工把守 gate。

Gói thành chỉ dẫn cho lần sau (user 23/09/2026)

打包为下次使用的指引（用户 23/09/2026）

Lớp lỗi chạm 
recurrence_threshold
(mặc định 3 — tức quá 2 lần) thì 
record()
TỰ gói ngay thành một thẻ chỉ dẫn:
bash
harness/metrics/guardrails/failure-<lớp>.md     # máy gói, có INDEX.md; được commit để phiên/máy khác cùng học
python3 harness/scripts/flywheel.py --kind failure --guardrails   # gói lại toàn bộ lớp đã đủ ngưỡng
Thẻ chỉ CHƯNG CẤT từ chính các dòng đã ghi (triệu chứng + cách sửa đã dùng + số lần + lần gần nhất) — không bịa luật mới. Hook 
session_start
in 5 thẻ mới nhất ở đầu MỖI phiên (mục 
🧯 [đã học]
), nên lần kế tiếp agent đọc được trước khi làm việc cùng loại. Thẻ mang trạng thái 
đã học, CHƯA duyệt
: nó là chỉ dẫn đọc-để-nhớ, KHÔNG cắn được ở CI. Muốn thành luật/skill chính thức thì vẫn 
--draft
rồi 
/propose
— cổng người giữ nguyên (RULE-02).
当失败类别达到 
recurrence_threshold
（默认 3——即超过 2 次）时，
record()
会立即自动打包成一张指引卡：
bash
harness/metrics/guardrails/failure-<lớp>.md     # máy gói, có INDEX.md; được commit để phiên/máy khác cùng học
python3 harness/scripts/flywheel.py --kind failure --guardrails   # gói lại toàn bộ lớp đã đủ ngưỡng
卡片仅摘录已记录的行本身（症状 + 已使用的修复方法 + 次数 + 最近一次），不会臆造新 rule。
session_start
hook 会在每个会话开头注入最近 5 张卡片（
🧯 [已学习]
章节），因此 Agent 下次在处理同类工作前即可读到。卡片状态为“已学习，尚未审批”：它是供阅读与记忆的指引，不会在 CI 中强制阻断。若要成为正式 rule/skill，仍需先运行 
--draft
，再运行 
/propose
——保持人工 gate（RULE-02）。

The adapter boundary (build-now-adapt-later)

Adapter 边界（build-now-adapt-later）

Everything above is deterministic and built now. The ONE quarantined unknown is 
harness/failure-flywheel.config.yaml
(
verified: false
): the recurrence threshold and taxonomy are best-guesses, and the "distill failures -> rule" model is absent. While it is unset, 
--draft
inserts a human-TODO stub instead of an auto-written rule. Finalize later by editing only that one file — calibrate the threshold, name a distill model, flip 
verified: true
.
上述所有内容都是确定性的，且已现在构建完成。唯一处于隔离状态的未知项是 
harness/failure-flywheel.config.yaml
（
verified: false
）：重复阈值和 taxonomy 均为最佳猜测，且“从 failures 提炼 rule”的模型尚不存在。在其配置完成前，
--draft
会插入 human-TODO stub，而不是自动编写 rule。后续只需编辑这一个文件即可最终确定——校准阈值、指定 distill model，并翻转为 
verified: true
。

Branches

分支

IDKindGuardHành viSkip / failureRejoin
B01conditional_requiredlớp lỗi ≥ 
recurrence_threshold
W03 
--draft
rồi W04		dưới ngưỡng → skip, chỉ reportW04
B02capability_optionalconfig 
verified: true
+ có distill model		stub có rule tự chưng cất thay vì human-TODOchưa verified → human-TODO stub (mặc định hiện tại)W04
ID类型守卫行为跳过/失败汇合点
B01conditional_required失败类别 ≥ 
recurrence_threshold
W03 
--draft
，然后 W04		低于阈值 → 跳过，仅执行 reportW04
B02capability_optionalconfig 
verified: true
+ 存在 distill model		stub 包含自动提炼的 rule，而非 human-TODO尚未 verified → human-TODO stub（当前默认）W04

Validation và stopping

验证与停止条件

Đếm và ngưỡng do 
failure-flywheel.py
quyết (tất định). Stub hợp lệ = OKF frontmatter + 
## Origin
, cố ý không 
## Plan
để qua cổng R7 như hạt giống. Luôn dừng ở W04 — không bao giờ tự promote.
计数和阈值由 
failure-flywheel.py
决定（确定性）。有效 stub = OKF frontmatter + 
## Origin
，有意不含 
## Plan
，从而以种子的形式通过 R7 gate。始终在 W04 停止——绝不自动 promote。

Examples

示例

  • Positive: agent bịa API lần thứ N (vượt ngưỡng) → 
    python3 harness/scripts/failure-flywheel.py record invented-api "called nonexistent fetchAll()"
    → 
    --report
    hiện 
    invented-api
    đánh dấu 
    DRAFT
    → 
    --draft invented-api
    viết 
    llmwiki/wiki/sources/draft/DDMMYY-failure-invented-api.md
    và dừng chờ 
    /propose
    .
  • Boundary/failure: 
    --draft invented-api --date 19/09/2026
    → exit 2 kèm lỗi rõ ràng, không viết file ngày sai.
  • 正向示例： Agent 第 N 次虚构 API（超过阈值）→ 
    python3 harness/scripts/failure-flywheel.py record invented-api "called nonexistent fetchAll()"
    → 
    --report
    将 
    invented-api
    标记为 
    DRAFT
    → 
    --draft invented-api
    写入 
    llmwiki/wiki/sources/draft/DDMMYY-failure-invented-api.md
    ，然后停止并等待 
    /propose
    。
  • 边界/失败： 
    --draft invented-api --date 19/09/2026
    → exit 2 并给出明确错误，不会写入日期错误的文件。

Related

相关内容

  • harness/scripts/code-logger.py
    — same by-code, gitignored-JSONL, fail-open capture pattern.
  • llmwiki/skills/dev-loop/propose.md
    — the gate this feeds.
  • llmwiki/skills/dev-loop/build-now-adapt-later.md
    — the quarantine pattern the config follows.
  • harness/scripts/code-logger.py
    ——相同的按 code 分类、gitignored-JSONL、fail-open 捕获模式。
  • llmwiki/skills/dev-loop/propose.md
    ——本流程接入的 gate。
  • llmwiki/skills/dev-loop/build-now-adapt-later.md
    ——config 所遵循的 quarantine 模式。