verify-claims

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Verify Claims

验证声明

The claim-to-evidence gate. Reviewers attack the gap between what a paper asserts and what it shows — an unbacked "we are the first", a "state-of-the-art" with no comparison, a "significant improvement" with no test, a speedup stated in the text that the table does not support. This skill finds those candidate claims, makes the author map each to its evidence, and produces a claims matrix (claim · location · evidence · status) so the gap is visible and closeable before a reviewer finds it.

It is the content counterpart to

verify-citations

: that skill proves each reference in the

.bib

resolves to a real paper; this skill proves each load-bearing sentence in the prose is backed by something in this paper (a number, a figure, a table, an experiment) or a real citation. Run both before any draft, rebuttal, or camera-ready leaves the machine.

这是声明与证据之间的核验关卡。审稿人会针对论文中声称与实际展示内容之间的差距提出质疑——比如无依据的“我们是首创”、未做对比的“state-of-the-art”、未经过测试的“显著改进”、正文所述的提速数据与表格不符等情况。本技能可找出这些待核查声明，让作者将每个声明对应到相关证据，并生成一份声明矩阵（声明·位置·证据·状态），以便在审稿人发现问题前，就能明确并填补这些差距。

它是

verify-citations

在内容层面的对应工具：前者用于验证

.bib

文件中的每个参考文献是否指向真实论文；而本技能用于验证正文中的每一句核心表述是否有本文中的内容（数据、图表、表格、实验）或真实引用作为支撑。在任何草稿、反驳意见回复或终稿提交前，都应运行这两个工具。

When to use

使用场景

The user asks to verify / audit / check claims (not citations): "are my claims supported?", "am I overclaiming?", "do my results match my tables?"
Before a rebuttal — reviewers' top complaint is unsupported or overclaimed contributions; close the gaps first, or arm the rebuttal with the evidence.
Before camera-ready or arXiv — last chance to soften an indefensible "first" or fix a number that drifted out of sync with a revised table.
After
```
polish-tables-figures
```
regenerated a table, to confirm the prose still matches the new numbers.

用户要求验证/审核/检查声明（而非引用）：比如“我的声明是否有支撑？”、“我是否过度声明了？”、“我的结果与表格数据是否一致？”
提交反驳意见前——审稿人最常抱怨的问题就是声明无支撑或过度夸大；先填补这些差距，或准备好证据用于反驳。
提交终稿或上传arXiv前——最后一次机会修正站不住脚的“首创”表述，或修复与更新后表格不一致的数据。
在
```
polish-tables-figures
```
工具重新生成表格后，确认正文内容仍与新数据匹配。

Inputs

输入

The paper's
```
.tex
```
(main file; the script follows
```
\input
```
/
```
\include
```
). Find it next to the
```
.bib
```
, or via the file with
```
\documentclass
```
.
The author, in the loop: the skill cannot decide whether a claim is true — it surfaces candidates and the author supplies (or admits the absence of) the evidence. Copilot, not pilot.
Optional:
```
.paper-memory/profile.yml
```
—
```
risk_appetite
```
and
```
contribution_type
```
set how hard the author wants to push novelty vs. hedge (see Memory).

论文的
```
.tex
```
主文件：脚本会识别
```
\input
```
/
```
\include
```
指令。该文件通常与
```
.bib
```
文件放在同一目录，或可通过包含
```
\documentclass
```
的文件找到。
参与流程的作者：本技能无法自行判断声明是否真实——它只会找出待核查的候选声明，由作者提供（或承认缺失）相关证据。它是辅助工具，而非主导者。
可选：
```
.paper-memory/profile.yml
```
文件——其中的
```
risk_appetite
```
（风险偏好）和
```
contribution_type
```
（贡献类型）可设置作者在创新性表述与保守表述之间的倾向（详见“记忆功能”）。

Process

流程

Extract candidate claims. Deterministic work belongs to the script — do not eyeball the paper for claims:
bash
```
python3 scripts/claim_audit.py path/to/main.tex --json /tmp/claims.json
```
It scans the prose (skipping math, comments, tables) for sentences carrying novelty markers ("first", "novel", "we are the only", "unlike prior work"), superiority markers ("outperforms", "state-of-the-art", "best", "superior"), magnitude/result markers ("significantly", "X% improvement", "Nx faster", "substantially"), and generalization markers ("always", "in all cases", "guarantees"). It also pulls every numeric token in the prose and every numeric cell in the tables, so you can cross-check. Each candidate gets a type, the sentence, and a
```
file:line
```
location.
Useful variants:
- ```
--type novelty
```
  (or
```
superiority
```
  ,
```
result
```
  ,
```
generalization
```
  ) — focus one class of claim.
- ```
--numbers
```
  — emit only the prose-number vs. table-number cross-check list (for the "do my results match my tables?" question).
- ```
--context N
```
  — include N sentences of surrounding text per claim.
- ```
--min-confidence high
```
  — only the strongest-signal candidates (fewer false positives) when the paper is large.
Exit codes:
```
0
```
no candidate claims found (rare — usually means the file parsed but is near-empty; check the input),
```
2
```
candidate claims were found (the normal case — they need author triage, not a "problem"),
```
1
```
operational failure (unreadable file, bad arguments). The script finds claims; it does not judge them — a nonzero
```
2
```
is the expected, healthy result, not a failure.
Map each candidate to evidence — with the author. This is the core of the skill and it is NOT automatable: the script flags "we achieve state-of-the-art accuracy"; only the author (or the paper's own Table 3) can say which result backs it. For each candidate, establish:
- Evidence — the specific result, table/figure number, experiment, theorem, or citation that supports it (a
```
\ref
```
  /
```
\label
```
  , a table cell, a section). "Section 4" is not evidence; "Table 3, row BERT-large, +2.1 F1 over the strongest baseline" is.
- Status —
```
SUPPORTED
```
  (evidence exists and matches),
```
WEAK
```
  (evidence exists but is thinner than the claim — e.g. "significantly" with no significance test),
```
UNSUPPORTED
```
  (no evidence found),
```
MISMATCH
```
  (prose number disagrees with the table), or
```
SCOPED
```
  (claim is fine once narrowed — see step 4).
Read references/claim-taxonomy.md for what each claim type requires as evidence and the standard reviewer attack on each.
Apply the overclaiming rules. Read references/overclaiming-rules.md. The high-frequency offenders:
- "first" / "novel" needs a defensible scope and a literature check — an absolute "first to X" is a single-counterexample-away from a desk-level embarrassment. Prefer a scoped "first to X under constraint Y".
- "state-of-the-art" / "outperforms" needs the comparison: which baselines, on which benchmark, by how much, and whether the baselines are current and fairly tuned.
- "significantly" / "substantial" is a statistical word: it needs a test (and the test named), not just a bigger mean. If no test was run, the word should change, not the data.
- Numbers in prose must equal the numbers in the tables/figures they summarize. The script's
```
--numbers
```
  mode lists prose vs. table numerics; reconcile every mismatch (a revised table that left a stale sentence behind is the classic camera-ready bug).
Decide the fix per claim, with the author. Three honest moves, never a fourth:
- Back it — add/point to the missing evidence (a result already in the paper, a citation, an experiment to run).
- Scope it — narrow the claim to what the evidence actually supports ("first" → "first under Y"; "always" → "in our experiments").
- Cut it — remove the claim if it cannot be backed or scoped. Never the fourth move: inventing evidence, a result, a citation, or a significance test that was not run. An honest "this claim is unsupported" beats a fabricated backing.
Write the claims matrix to
```
paper-workspace/review/claims-matrix.md
```
: one row per load-bearing claim — claim · location · type · evidence · status · suggested fix. Lead the chat summary with counts by status and the most dangerous open items (UNSUPPORTED and MISMATCH first). Append the run to
```
INDEX.md
```
.
Re-run after fixes. Once the author edits, re-run
```
claim_audit.py
```
(and
```
--numbers
```
) to confirm softened/scoped claims no longer trip the markers and no prose/table mismatch remains. Stop when every load-bearing claim is SUPPORTED, SCOPED, or an explicitly author-accepted WEAK — not on an open-ended "keep improving" loop (cap at a couple of passes; report what remains open).

提取候选声明：确定性的工作由脚本完成——无需人工逐句查找声明：
bash
```
python3 scripts/claim_audit.py path/to/main.tex --json /tmp/claims.json
```
脚本会扫描正文内容（跳过公式、注释、表格），识别带有以下标记的句子：创新性标记（“first”、“novel”、“we are the only”、“unlike prior work”）、优越性标记（“outperforms”、“state-of-the-art”、“best”、“superior”）、结果/量级标记（“significantly”、“X% improvement”、“Nx faster”、“substantially”），以及泛化性标记（“always”、“in all cases”、“guarantees”）。同时，它还会提取正文中的所有数字内容和表格中的所有数字单元格，以便交叉核对。每个候选声明都会被标记类型、原文句子和
```
文件:行号
```
的位置信息。
实用参数变体：
- ```
--type novelty
```
  （或
```
superiority
```
  、
```
result
```
  、
```
generalization
```
  ）——聚焦某一类声明。
- ```
--numbers
```
  ——仅输出正文数字与表格数字的交叉核对列表（用于解决“我的结果与表格数据是否一致？”的问题）。
- ```
--context N
```
  ——为每个声明附带N句上下文内容。
- ```
--min-confidence high
```
  ——当论文篇幅较长时，仅输出信号最强的候选声明（减少误报）。
退出码说明：
```
0
```
表示未找到候选声明（罕见情况，通常意味着文件解析成功但内容近乎为空，请检查输入）；
```
2
```
表示找到候选声明（正常情况，需要作者进行分类处理，并非“问题”）；
```
1
```
表示操作失败（文件无法读取、参数错误）。脚本仅负责发现声明，不负责判断声明——非零的
```
2
```
是预期的正常结果，而非失败。
将每个候选声明与证据关联——需作者参与：这是本技能的核心环节，且无法自动化：脚本标记出“我们实现了state-of-the-art准确率”这样的句子，但只有作者（或论文中的表3）能指出具体哪项结果为其提供支撑。针对每个候选声明，需确认：
- 证据：支撑该声明的具体结果、图表/表格编号、实验、定理或引用（比如
```
\ref
```
  /
```
\label
```
  、表格单元格、章节）。“第4节”不属于有效证据；“表3中BERT-large行，比最强基线高出2.1的F1值”才是有效证据。
- 状态：
```
SUPPORTED
```
  （有匹配的证据支撑）、
```
WEAK
```
  （有证据但支撑力度不足——比如使用“significantly”却未进行显著性检验）、
```
UNSUPPORTED
```
  （未找到证据）、
```
MISMATCH
```
  （正文数字与表格数据不符），或
```
SCOPED
```
  （缩小范围后声明合理——详见步骤4）。
阅读references/claim-taxonomy.md了解各类声明所需的证据标准，以及审稿人针对各类声明的常见质疑点。
应用过度声明规则：阅读references/overclaiming-rules.md。高频违规情况包括：
- “first”/“novel”：需要明确的适用范围和文献调研——绝对化的“首个实现X”只需一个反例就会让作者陷入尴尬。建议使用限定范围的表述，比如“首个在约束条件Y下实现X”。
- “state-of-the-art”/“outperforms”：需要明确对比对象：对比了哪些基线模型、在哪个基准数据集上、提升幅度多少，以及基线模型是否是当前最新且经过公平调优的。
- “significantly”/“substantial”：属于统计学术语：需要进行显著性检验（并明确检验方法），而非仅依赖均值差异。如果未进行检验，应修改表述，而非篡改数据。
- 正文中的数字必须与表格/图表中的对应数字一致。脚本的
```
--numbers
```
  模式会列出正文与表格的数字对比；需解决所有不一致问题（修改表格后未更新正文是终稿阶段的典型错误）。
针对每个声明确定修正方案——需作者参与：有三种诚实的处理方式，绝不能选择第四种：
- 补充支撑：添加/指向缺失的证据（论文中已有的结果、引用、待开展的实验）。
- 缩小范围：将声明限定在证据实际支撑的范围内（比如“首创”→“在Y约束下首创”；“始终”→“在我们的实验中”）。
- 删除声明：如果声明无法获得支撑或缩小范围，则删除该声明。绝不能选择第四种方式：编造未存在的证据、结果、引用或显著性检验来“支撑”声明。诚实承认“该声明无支撑”远胜于编造证据。
将声明矩阵写入
paper-workspace/review/claims-matrix.md
：每一行对应一个核心声明——声明·位置·类型·证据·状态·建议修正方案。在聊天总结中先按状态统计数量，并列出最具风险的未解决问题（优先列出UNSUPPORTED和MISMATCH）。将本次运行记录添加到
```
INDEX.md
```
中。
修正后重新运行：作者完成修改后，重新运行
```
claim_audit.py
```
（包括
```
--numbers
```
模式），确认经过弱化/范围限定的声明不再触发标记，且正文与表格无数据不一致问题。当所有核心声明的状态为SUPPORTED、SCOPED，或作者明确接受的WEAK时即可停止——无需无限循环优化（最多进行几次迭代；报告剩余未解决问题即可）。

Output

输出

```
claim_audit.py
```
stdout (and
```
--json
```
): candidate claims with type, location, and the prose-vs-table numeric cross-check — a worklist, not a verdict.
```
paper-workspace/review/claims-matrix.md
```
: the reviewable claims matrix the author fills in and acts on, plus a one-line
```
INDEX.md
```
entry.
A chat summary: counts by status, the riskiest open claims, and the per-claim fix (back / scope / cut) — never a fabricated backing.

```
claim_audit.py
```
的标准输出（及
```
--json
```
格式输出）：包含候选声明的类型、位置，以及正文与表格数字的交叉核对结果——这是一份待处理清单，而非最终结论。
```
paper-workspace/review/claims-matrix.md
```
：可供作者填写并执行的可审核声明矩阵，以及在
```
INDEX.md
```
中的一行记录。
聊天总结：按状态统计的数量、最具风险的未解决声明，以及针对每个声明的修正方案（补充支撑/缩小范围/删除）——绝不能包含编造的证据。

Hard rules

硬性规则

The skill never decides a claim is true or false on its own. It has no oracle for "is this the first paper to do X" or "is this really state-of-the-art" — it surfaces the candidate and routes the judgment to the author and the paper's own evidence. Do not use the model's self-assessment as the verification signal (it measures plausibility, not correctness, and is worst exactly when most confident).
Never fabricate evidence. No invented result, table number, citation, baseline, or significance test to "support" a claim. If it cannot be backed, it must be scoped or cut.
Numbers are facts, not prose. A prose number that disagrees with its table is a MISMATCH to reconcile from the table (or the underlying result), never by editing the table to match a sentence the author likes better.
"first"/"SOTA"/"significant" are load-bearing words with evidentiary cost. Treat them as claims requiring proof, not rhetorical flourish.
This skill checks claim→evidence within this paper; it does not verify the
```
.bib
```
resolves — that is
```
verify-citations
```
. When a claim's evidence is a citation, hand that reference to verify-citations.
It reports and explains; it never edits the paper or submits anything. The author makes every back/scope/cut decision.

本技能绝不自行判断声明的真假：它无法判断“这是否是首个实现X的论文”或“这是否真的是state-of-the-art”——它只会找出候选声明，将判断权交给作者和论文本身的证据。不要将模型的自我评估作为验证依据（模型评估的是合理性，而非正确性，且在最自信时准确率最低）。
绝不能编造证据：不得编造未存在的结果、表格编号、引用、基线模型或显著性检验来“支撑”声明。如果声明无法获得支撑，必须缩小范围或删除。
数字是事实，而非表述内容：正文数字与表格数字不一致属于MISMATCH，需根据**表格（或底层结果）**进行修正，绝不能为了匹配作者偏好的句子而修改表格。
“first”/“SOTA”/“significant”是需要证据支撑的核心词汇：应将它们视为需要验证的声明，而非修辞手段。
本技能核查的是本文内声明与证据的对应关系；它不负责验证
```
.bib
```
文件中的引用是否有效——这是
```
verify-citations
```
的功能。当声明的证据是引用时，应将该引用交给verify-citations工具处理。
本技能仅负责报告和解释，绝不直接修改论文或提交任何内容。所有补充支撑/缩小范围/删除的决策均由作者做出。

Adapt to your discipline

适配不同学科

Defaults target CS venues (IEEE/ACM/ML). The marker lexicons and evidence expectations are field-specific: a theory paper's claims trace to theorems and proofs (not tables); an HCI paper's to study design and significance reporting; a survey's to coverage and taxonomy completeness rather than "outperforms". Edit

references/claim-taxonomy.md

and the marker lists in

claim_audit.py

(documented inline) for your field's claim vocabulary and standards of proof.

默认设置针对计算机科学领域的会议（IEEE/ACM/ML）。标记词汇和证据要求因领域而异：理论论文的声明需追溯到定理和证明（而非表格）；人机交互（HCI）论文的声明需对应研究设计和显著性报告；综述论文的声明需关注覆盖范围和分类完整性，而非“性能超越”。可编辑

references/claim-taxonomy.md

和

claim_audit.py

中的标记列表（内有文档说明），以适配所在领域的声明词汇和验证标准。

Bundled resources

附带资源

```
scripts/claim_audit.py
```
— extracts candidate claim sentences and the prose/table numeric cross-check from a
```
.tex
```
. Stdlib only;
```
--help
```
for all options. Run it; do not hand-scan the paper.
references/claim-taxonomy.md — the claim types, what evidence each needs, and the standard reviewer attack on each.
references/overclaiming-rules.md — the high-frequency overclaims ("first", "SOTA", "significant", stale numbers) and the back/scope/cut remedy for each.

```
scripts/claim_audit.py
```
：从
```
.tex
```
文件中提取候选声明句子，并生成正文与表格数字的交叉核对结果。仅依赖标准库；使用
```
--help
```
查看所有参数。请运行该脚本，不要人工逐句扫描论文。
references/claim-taxonomy.md：声明类型、各类声明所需的证据标准，以及审稿人针对各类声明的常见质疑点。
references/overclaiming-rules.md：高频过度声明类型（“first”、“SOTA”、“significant”、过时数字），以及对应的补充支撑/缩小范围/删除的修正方案。

Memory

记忆功能

Uses the shared

.paper-memory/

convention (full spec:

paper-memory-convention.md

At start: read
```
lessons.md
```
to skip re-flagging claims the author already scoped or backed this cycle, and lead with any
```
recurring
```
overclaiming habit (e.g. "tends to write absolute 'first' claims; scope them up front"). Read
```
profile.yml
```
```
risk_appetite
```
—
```
conservative
```
authors want every WEAK claim hedged;
```
aggressive
```
authors accept defensible WEAK claims they will fight for in review.
At end: append durable findings via
```
reflect-and-improve
```
's
```
reflect_log.py append
```
in the shared format
```
- [YYYY-MM-DD] (verify-claims | <scope>) issue -> recommendation
```
. A habit across the paper or across papers (e.g. unscoped novelty claims, "significant" without a test) is
```
recurring
```
; a single fixed sentence is
```
this-paper
```
. Never record a fabricated backing — only the pattern and the honest fix.
Create
```
.paper-memory/
```
on demand if absent and offer to add it to the project
```
.gitignore
```
. Local-only; never uploaded or copied into this repo.

遵循共享的

.paper-memory/

约定（完整规范：

paper-memory-convention.md

）。

开始时：读取
```
lessons.md
```
，跳过已经在本轮中被作者缩小范围或补充支撑的声明，并优先指出反复出现的过度声明习惯（比如“倾向于使用绝对化的‘first’声明；建议预先缩小范围”）。读取
```
profile.yml
```
中的
```
risk_appetite
```
——
```
conservative
```
（保守型）作者希望所有WEAK声明都进行弱化处理；
```
aggressive
```
（激进型）作者接受可辩护的WEAK声明，并愿意在评审阶段为之争辩。
结束时：通过
```
reflect-and-improve
```
的
```
reflect_log.py append
```
工具，以共享格式
```
- [YYYY-MM-DD] (verify-claims | <范围>) 问题 -> 建议
```
添加持久化的发现。如果是整篇论文或多篇论文中反复出现的习惯（比如无范围限定的创新性声明、使用“significant”却未进行检验），标记为
```
recurring
```
；如果是单个已修正的句子，标记为
```
this-paper
```
。绝不能记录编造的证据——仅记录模式和诚实的修正方案。
如果
```
.paper-memory/
```
目录不存在，会自动创建，并建议将其添加到项目的
```
.gitignore
```
中。该目录仅本地存在；绝不会上传或复制到本仓库。