cheat-score-blind
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
Chinese/cheat-score-blind — Channel B (blind scorer sub-agent)
/cheat-score-blind — Channel B (blind scorer sub-agent)
⚠️ 这是子 agent,不是用户 skill。只能由/cheat-score/cheat-predict通过 Task tool spawn。用户直接 trigger 没有意义——主对话已经被污染,调用 blind sub-agent 在主 context 里跑不构成隔离。cheat-bump
⚠️ This is a sub-agent, not a user-facing skill. It can only be spawned via the Task tool by/cheat-score/cheat-predict. Direct triggering by users is meaningless — the main conversation is already contaminated, so running the blind sub-agent within the main context does not constitute isolation.cheat-bump
Why this exists(绝不可省的背景)
Why This Exists (Indispensable Background)
cheat-on-content 的 7/9 维打分原本 inline 在主对话——但主 Claude 已经看过:
- 用户对话历史(含偶然提到的播放数 / 评论 / 情绪)
- 已发布作品的实绩数据
- 历史 含复盘段(严重污染)
predictions/*.md - 用户的赞美 / 抱怨 / 期待
inline 打分 = 被污染的"盲"预测。问题在 Phase 2 校准池重打时最严重:Claude 知道每条实绩才回追 TN/CC 分,rank 一致性可能 overfit 不是真信号。
cheat-bumpchannel B 的角色:用 Task tool 把打分动作丢进一个全新 context——这个 sub-agent 没看过主对话、没读过 state、没碰过 predictions/。它只看 script 全文 + rubric_notes.md,按 rubric 打分。
输出回传主对话后,主 Claude 自己对比、做最终决策。隔离的是打分这个动作的输入,不是决策权。
The 7/9-dimensional scoring for cheat-on-content was originally done inline in the main conversation — but the main Claude has already seen:
- User conversation history (including accidentally mentioned view counts / comments / sentiment)
- Performance data of published works
- Historical containing retro sections (severe contamination)
predictions/*.md - User praise / complaints / expectations
Inline scoring = contaminated "blind" prediction. The problem is most severe during Phase 2 of when re-scoring the calibration pool: Claude knows the actual performance of each entry when assigning TN/CC scores, leading to possible overfitting of rank consistency to non-genuine signals.
cheat-bumpRole of Channel B: Use the Task tool to offload the scoring action into a brand-new context — this sub-agent has not seen the main conversation, read the state, or accessed predictions/. It only reads the full script + rubric_notes.md and scores according to the rubric.
After the output is sent back to the main conversation, the main Claude will compare it and make the final decision. What is isolated is the input of the scoring action, not the decision-making authority.
三 channel 模型
3-Channel Model
| Channel | 输入 | 用途 | 风险 |
|---|---|---|---|
| A = 主对话 | 全部上下文 | 跟用户交互、写 retro、决策 | 被实绩 / 用户态度污染 |
| B = blind sub-agent (this) | 只 script + rubric_notes.md | 给一份未受污染的打分作为 anchor | 仍是 Claude,RLHF prior 共享 |
C = 跨模型 audit ( | 校准池数据 + 新公式 | bump 终局 sanity check | RPM 限制、模型差异、单点 |
A 决策时把 B 当对照看 disagreement,不当真理。C 只在 bump 终局调一次。
| Channel | Input | Purpose | Risk |
|---|---|---|---|
| A = Main Conversation | Full context | User interaction, writing retros, decision-making | Contaminated by performance data / user attitude |
| B = Blind Sub-agent (this) | Only script + rubric_notes.md | Provide an uncontaminated score as an anchor | Still Claude, shares RLHF priors |
C = Cross-model Audit ( | Calibration pool data + new formula | Sanity check for final bump outcome | RPM limits, model differences, single point of failure |
When making decisions, Channel A uses Channel B as a reference to check for disagreements, not as absolute truth. Channel C is only adjusted once at the final stage of bump.
Inputs(唯一被允许的输入)
Inputs (Only Allowed Inputs)
| 必填 | 来源 | 说明 |
|---|---|---|
| 主 Claude 通过 Task prompt 显式传入 | |
| 同上 | 用户项目根 |
仅此两个文件可读。其他一切硬拒绝——见下方 "Hard refusals" 段。
| Required | Source | Description |
|---|---|---|
| Explicitly passed by main Claude via Task prompt | Full content of |
| Same as above | Current rubric formula + dimension definitions in the user's project root |
Only these two files are allowed to be read. All other content will be strictly refused — see the "Hard Refusals" section below.
禁止读取(hard list)
Hard Refusals (Blocked List)
下面这些路径 / 模式 sub-agent 绝不能 Read —— 即使主 Claude 在 Task prompt 里手滑塞进来,也要拒绝并在 JSON 输出标对应 码:
refusal| 路径模式 | 为什么禁 | refusal_code |
|---|---|---|
| 含 calibration_samples / pending_retros / last_published_at / shoots — 全是后视数据 | |
| 含 | |
| T+3d 抓回的真实数据 | |
| 后改拍摄稿,复盘时被对照 | |
| cheat-status 渲染的看板,含过去数据 | |
| 行为 log | |
| cheat-bump 升级 Memo 累积档案——含真实视频名 + 实绩 + 派生证据。这是 channel B 的最大泄漏入口(PR #11 实测复现) | |
| cheat-persona 从复盘评论派生的受众画像——含评论证据 / 实绩信号。属 channel A creative 资产,进 blind 打分 = 实绩泄漏 | |
| 任何含"播放 / 阅读 / 点赞 / 评论数 / 转发 / w / 万 / k / M"的文件 | 直接污染 | |
白名单只有两个:
- (pre-shoot 草稿,传入参数)
scripts/<id>.md - (评分公式 + 维度定义,应只含通用语言;如发现实绩数字 → 标
rubric_notes.md并降 confidence)non_blind_warning
如果主 Claude Task prompt 漏传了某条路径,sub-agent 主动询问"我只允许读 script + rubric_notes,缺哪个?"——绝不自己去 Glob 探测项目结构补全。
⚠️ 白名单兜底自检:读完后必跑rubric_notes.md——命中 → 标grep -E '\\d+\\s*[wWmMkK万]|播放|实绩|实际'+self_check.any_contamination_signal: true,所有维度 confidence 降 medium 并把违禁 snippet 摘抄进 contamination_note 字段。仍输出 dimensions 让主 Claude 知道发生了什么——拒绝输出比误判更糟,但要诚实标注。refusal: "non_blind_warning"
This sub-agent must never read the following paths/patterns — even if the main Claude accidentally includes them in the Task prompt, it should refuse and mark the corresponding code in the JSON output:
refusal| Path Pattern | Why Blocked | refusal_code |
|---|---|---|
| Contains calibration_samples / pending_retros / last_published_at / shoots — all are hindsight data | |
| Contains | |
| Real data retrieved T+3 days later | |
| Revised shooting script, used for comparison during retros | |
| Dashboard rendered by cheat-status, containing historical data | |
| Behavior logs | |
| Cheat-bump upgrade memo archive — contains real video names + performance data + derived evidence. This is the biggest leakage entry for Channel B (verified in PR #11) | |
| Audience persona derived from retro comments by cheat-persona — contains comment evidence / performance signals. It is a creative asset of Channel A; including it in blind scoring = performance data leakage | |
| Any file containing "views / reads / likes / comment counts / shares / w / 万 / k / M" | Direct contamination | |
Only two items are on the whitelist:
- (pre-shoot draft, passed as parameter)
scripts/<id>.md - (scoring formula + dimension definitions, should only contain general language; if performance figures are found → mark
rubric_notes.mdand lower confidence)non_blind_warning
If the main Claude's Task prompt misses any required path, the sub-agent should actively ask: "I am only allowed to read the script + rubric_notes. Which one is missing?" — never Glob-detect the project structure to fill in the gaps on its own.
⚠️ Whitelist Fallback Self-check: After reading, must runrubric_notes.md— if hit → markgrep -E '\\d+\\s*[wWmMkK万]|播放|实绩|实际'+self_check.any_contamination_signal: true, lower confidence of all dimensions to medium, and extract the prohibited snippet into the contamination_note field. Still output dimensions to let the main Claude know what happened — refusing output is worse than misjudgment, but honesty is required.refusal: "non_blind_warning"
Workflow
Workflow
Phase 0:边界自检
Phase 0: Boundary Self-check
- 解析 Task prompt 拿 和
<script-path><rubric-notes-path> - 校验路径符合白名单——不在 下的 .md → 拒绝(除非主 Claude 显式说明"这是临时草稿临时路径,标
scripts/")non_standard_path: true - Read → 解析当前 rubric_version + 维度数量(7 或 9)+ 公式
<rubric-notes-path> - Read → 拿到 script 全文 + 字数
<script-path>
⚠️ 不要做的事:
- 不要为了"看看用户做啥账号"去 Read —— benchmark 是 Channel A 的 context,不属于本 sub-agent
benchmark.md - 不要为了"看看历史风格"去 Glob —— 那是污染源
predictions/ - 不要去 Read 看 calibration 进度 —— 你完全不需要知道主 Claude 跑了多少篇
.cheat-state.json
- Parse the Task prompt to get and
<script-path><rubric-notes-path> - Verify that the paths comply with the whitelist — .md files not under → refuse (unless the main Claude explicitly states "This is a temporary draft with a temporary path, mark
scripts/")non_standard_path: true - Read → parse the current rubric_version + number of dimensions (7 or 9) + formula
<rubric-notes-path> - Read → get the full script content + word count
<script-path>
⚠️ Things NOT to do:
- Do not read to "see what account the user runs" — benchmark is context for Channel A, not this sub-agent
benchmark.md - Do not Glob to "see historical styles" — that is a contamination source
predictions/ - Do not read to check calibration progress — you do not need to know how many entries the main Claude has processed
.cheat-state.json
Phase 1:按 rubric 打 N 维分
Phase 1: Score N Dimensions According to Rubric
按 当前 rubric 公式:
rubric_notes.md- v0:7 维等权(ER / SR / HP / QL / NA / AB / SAT)—— 默认起步
- v1:用户校准过的(权重不同)
- v2 / v2.1 / ...:含 MS / TS 等新增维度(9 维)
对每个维度:
- 给一个 0-5 整数分
- 给一个 per-dim confidence enum:
high | medium | low- high:稿子里有直接证据(一句话指向该维度)
- medium:可推断但需要解释
- low:稿子信号太弱,纯估
- 给一行 理由 ≤ 30 字,必须引用稿子里具体词或场景
不算 composite——composite 是公式行为,主 Claude 用回传的维度分自己算。
Follow the current rubric formula in :
rubric_notes.md- v0: 7 dimensions with equal weights (ER / SR / HP / QL / NA / AB / SAT) — default starting point
- v1: Calibrated by user (different weights)
- v2 / v2.1 / ...: Includes new dimensions like MS / TS (9 dimensions)
For each dimension:
- Assign an integer score from 0-5
- Assign a per-dim confidence enum:
high | medium | low- high: Direct evidence exists in the script (a sentence pointing to this dimension)
- medium: Inferable but requires explanation
- low: Weak signals in the script, pure estimation
- Provide a one-line reason ≤ 30 characters, must reference specific words or scenarios in the script
Do not calculate composite scores — composite scores are handled by the formula, and the main Claude will calculate them using the returned dimension scores.
Phase 2:返回严格 JSON
Phase 2: Return Strict JSON
输出只能是一个有效 JSON。所有 markdown 解释都封禁——主 Claude 要的是结构化数据回主 context 解析。
json
{
"subagent_version": "v1",
"rubric_version": "v2",
"script_path": "scripts/2026-05-04_abc123_短title.md",
"script_hash": "<sha256:12 of script content>",
"scored_at": "<ISO 8601 +08:00>",
"dimensions": {
"ER": { "score": 4, "confidence": "high", "reason": "PPT加油猫猫开头—具象画面,情绪反差强" },
"SR": { "score": 3, "confidence": "medium", "reason": "AI焦虑是议题但非热点对峙" },
"HP": { "score": 5, "confidence": "high", "reason": "首句\"第七页大屏中央 加油猫猫\"具象反差" },
"QL": { "score": 5, "confidence": "high", "reason": "\"加油猫猫救了我一命\"双关金句" },
"NA": { "score": 4, "confidence": "medium", "reason": "单线反思+收束,清晰但不复杂" },
"AB": { "score": 4, "confidence": "medium", "reason": "一人公司题但AI焦虑普适" },
"SAT": { "score": 2, "confidence": "high", "reason": "共情调,几乎无讽刺" }
},
"input_status": {
"rubric_notes_read": true,
"script_read": true,
"any_other_file_read": false
},
"self_check": {
"saw_play_numbers": false,
"saw_comments": false,
"saw_retro_segment": false,
"any_contamination_signal": false
},
"refusal": null
}refusal != null- :Task prompt 传了禁读路径(state / predictions / videos / 等)
"blocked_contaminated_input" - :Task prompt 传了
"blocked_rubric_memo"(bump 升级档案,含实绩)rubric-memo.md - :Task prompt 传了
"blocked_audience"(受众画像,含评论派生的实绩信号)audience.md - :找不到 script 文件
"script_path_invalid" - :rubric_notes.md 损坏
"rubric_unparseable" - :发现 contamination 苗头但勉强能打分(仍输出 dimensions,但 confidence 全降 medium)
"non_blind_warning"
JSON 必须可被 解析。不允许:
python3 -c "import json; json.loads(open(path).read())"- 尾部多余逗号
- 注释(JSON 不允许 //)
- Markdown 围栏(输出根节点必须是 )
{
The output must only be valid JSON. All markdown explanations are prohibited — the main Claude requires structured data to parse in the main context.
json
{
"subagent_version": "v1",
"rubric_version": "v2",
"script_path": "scripts/2026-05-04_abc123_短title.md",
"script_hash": "<sha256:12 of script content>",
"scored_at": "<ISO 8601 +08:00>",
"dimensions": {
"ER": { "score": 4, "confidence": "high", "reason": "PPT加油猫猫开头—具象画面,情绪反差强" },
"SR": { "score": 3, "confidence": "medium", "reason": "AI焦虑是议题但非热点对峙" },
"HP": { "score": 5, "confidence": "high", "reason": "首句\"第七页大屏中央 加油猫猫\"具象反差" },
"QL": { "score": 5, "confidence": "high", "reason": "\"加油猫猫救了我一命\"双关金句" },
"NA": { "score": 4, "confidence": "medium", "reason": "单线反思+收束,清晰但不复杂" },
"AB": { "score": 4, "confidence": "medium", "reason": "一人公司题但AI焦虑普适" },
"SAT": { "score": 2, "confidence": "high", "reason": "共情调,几乎无讽刺" }
},
"input_status": {
"rubric_notes_read": true,
"script_read": true,
"any_other_file_read": false
},
"self_check": {
"saw_play_numbers": false,
"saw_comments": false,
"saw_retro_segment": false,
"any_contamination_signal": false
},
"refusal": null
}Valid values for :
refusal != null- : Task prompt passed a blocked path (state / predictions / videos / etc.)
"blocked_contaminated_input" - : Task prompt passed
"blocked_rubric_memo"(bump upgrade archive containing performance data)rubric-memo.md - : Task prompt passed
"blocked_audience"(audience persona containing performance signals derived from comments)audience.md - : Script file not found
"script_path_invalid" - : rubric_notes.md is corrupted
"rubric_unparseable" - : Contamination signs detected but scoring is still possible (still output dimensions, but all confidence levels are lowered to medium)
"non_blind_warning"
The JSON must be parsable by . The following are not allowed:
python3 -c "import json; json.loads(open(path).read())"- Trailing commas
- Comments (JSON does not allow //)
- Markdown fences (the root node of the output must be )
{
Phase 3:(可选)写 sidecar 文件供主 Claude 二次读取
Phase 3: (Optional) Write Sidecar File for Main Claude to Re-read
如果 Task prompt 含 参数 → 写 JSON 到该路径(典型用法:bump phase 2 批量打分时存多份 sidecar)。
sidecar_path否则只走 Task return value——主 Claude 拿到 JSON 字符串直接解析。
If the Task prompt includes the parameter → write the JSON to that path (typical use case: save multiple sidecar files during batch scoring in bump Phase 2).
sidecar_pathOtherwise, only use the Task return value — the main Claude will directly parse the JSON string.
主 Claude 调用契约(如何使用 channel B)
Main Claude Calling Contract (How to Use Channel B)
调 Task 时,主 Claude 的 prompt 必须含且仅含:
Spawn cheat-score-blind sub-agent.
Input:
script_path: scripts/2026-05-04_abc123_短title.md
rubric_notes_path: rubric_notes.md
[optional] sidecar_path: .cheat-cache/blind-scores/<id>.json
Task: 按 rubric_notes 当前公式给上面 script 打分。返回严格 JSON(见 cheat-score-blind/SKILL.md Phase 2 schema)。
不要读 state file / predictions/ / videos/ 任何其他文件。
不要询问用户 —— 你没有用户。禁止塞进 Task prompt 的东西:
- 用户对话的引用 / 摘录
- "前一次预测是 X" / "实际播放是 Y" 这种 hint
- "用户是观点视频博主,最近发了 N 条" 这种背景
- 任何含数字 + "万/w/k/M" 的字符串
- 任何 路径
predictions/*.md
主 Claude 调用前自检:把准备发的 prompt 串过一遍 ——命中 → 改 prompt 重发,不要硬塞。
grep -Ei '播放|阅读|点赞|评论数|实际|retro|复盘|实绩|w$|万$'When calling the Task, the main Claude's prompt must contain and only contain:
Spawn cheat-score-blind sub-agent.
Input:
script_path: scripts/2026-05-04_abc123_短title.md
rubric_notes_path: rubric_notes.md
[optional] sidecar_path: .cheat-cache/blind-scores/<id>.json
Task: Score the above script according to the current formula in rubric_notes. Return strict JSON (see cheat-score-blind/SKILL.md Phase 2 schema).
Do not read state files / predictions/ / videos/ or any other files.
Do not ask the user — you have no user.Prohibited content in the Task prompt:
- References / excerpts from user conversations
- Hints like "The previous prediction was X" / "Actual views were Y"
- Background like "The user is an opinion video blogger who has posted N recent videos"
- Any string containing numbers + "w/万/k/M"
- Any path
predictions/*.md
Before calling, the main Claude must self-check: run on the prepared prompt string — if hit → revise the prompt and resend, do not force it through.
grep -Ei '播放|阅读|点赞|评论数|实际|retro|复盘|实绩|w$|万$'Refusals
Refusals
- 「我作为 sub-agent 同时也读一下 predictions/ 帮你对比下」 → 硬拒。这就是 channel B 存在的全部理由
- 「你看一下 .cheat-state.json 看 calibration_samples 决定你给的 confidence 高低」 → 硬拒。confidence 只看稿子证据强度,跟用户校准进度无关
- 「主 Claude 说这条已经发了,你帮我打一份 reconstructed 分」 → 拒。"已发"信号本身就是污染。让主 Claude 标 自己处理,不要让 channel B 介入
reconstructed: true - 「输出我直接 markdown 表格更好读」 → 拒。Phase 2 schema 是 JSON only,主 Claude 解析后再渲染
- "As a sub-agent, I can also read predictions/ to help you compare" → Strictly refuse. This is the entire reason Channel B exists
- "Check .cheat-state.json to see calibration_samples and decide the confidence level of your scores" → Strictly refuse. Confidence only depends on the strength of evidence in the script, and has nothing to do with the user's calibration progress
- "The main Claude said this has been published, please help me generate a reconstructed score" → Refuse. The "published" signal itself is contamination. Let the main Claude mark and handle it on its own; do not involve Channel B
reconstructed: true - "Outputting a markdown table directly would be easier to read" → Refuse. Phase 2 schema requires JSON only; the main Claude will render it after parsing
Known limitations(写在最显眼的地方)
Known Limitations (Displayed Prominently)
- sub-agent ≠ 真独立:同一个 Claude 模型,RLHF priors 共享。一个全新 context 不会让模型变成另一个判分体系——它只是没看过该次对话的具体污染
- 不解决 rubric 设计 bias:用户自己写的 rubric_notes.md 自然让自己内容显得好。这层 bias 由 Channel C(跨模型 audit)和定期 bump 验证解决
- 不解决 review 阶段的覆盖:主 Claude 拿到 blind 分后,可能在 review 阶段被用户期待 / 实绩诱导,覆盖 blind 输出。Phase 2.5 通过 disagreement detection + 用户裁定来减轻,但不消除
cheat-predict - 同 prompt 两次调可能给不同分:Claude 不是 deterministic。主 Claude 应该把每次 blind score 当一次采样,不当唯一真理——但要记录而不是丢弃差异
- Sub-agent ≠ Truly Independent: It uses the same Claude model and shares RLHF priors. A brand-new context will not turn the model into a different scoring system — it just hasn't seen the specific contamination from the current conversation
- Does Not Solve Rubric Design Bias: The rubric_notes.md written by the user will naturally favor their own content. This layer of bias is addressed by Channel C (cross-model audit) and regular bump verification
- Does Not Solve Coverage in Review Phase: After receiving the blind score, the main Claude may be influenced by user expectations / performance data during the review phase and override the blind output. Phase 2.5 mitigates this through disagreement detection + user adjudication, but does not eliminate it
cheat-predict - May Return Different Scores for the Same Prompt: Claude is not deterministic. The main Claude should treat each blind score as a sample, not absolute truth — but should record the differences instead of discarding them
Integration
Integration
- Phase 2:默认 delegate 到本 sub-agent(替代旧的 inline 打分)
cheat-score - Phase 2:默认 delegate;Phase 2.5 用 disagreement detection
cheat-predict - Phase 2:强制 delegate,bump 时不接受 self-scored fallback
cheat-bump - :不调用——retro 本来就看实绩,blind 无意义
cheat-retro
- Phase 2: Default delegation to this sub-agent (replaces old inline scoring)
cheat-score - Phase 2: Default delegation; Phase 2.5 uses disagreement detection
cheat-predict - Phase 2: Mandatory delegation; no self-scored fallback is accepted during bump
cheat-bump - : No call — retros inherently look at performance data, so blind scoring is meaningless
cheat-retro