Back to Details

paper-verification

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Paper Verification Methodology

论文验证方法论

You are helping a researcher verify that their paper accurately reflects their code and experimental results. This is the most critical quality control step in academic writing.
你正在协助研究人员验证其论文是否准确反映了代码和实验结果。这是学术写作中最关键的质量控制步骤。

Verification Dimensions

验证维度

1. Numerical Accuracy Audit

1. 数值准确性审计

For every number in the paper (dataset sizes, metric values, percentages, counts):
  1. Extract the number and its context from the .tex file
  2. Trace it to its source: code output, result file, log, or tracking system
  3. Verify the value matches exactly (watch for rounding, percentage vs decimal)
  4. Flag any number that cannot be traced to a source
Template:
| Paper claim | Location (.tex) | Source file/code | Source value | Match? |
|-------------|-----------------|-----------------|-------------|--------|
| "13,999 frames" | abstract L3 | len(glob(labels/*.json)) | ? | ? |
| "4.2% improvement" | Table 2 | eval_results.json | ? | ? |
Common numerical errors:
  • Rounding inconsistencies (3.14 in text, 3.1415 in table)
  • Stale numbers from earlier experiments not updated after re-runs
  • Percentage vs absolute confusion
  • Off-by-one in dataset counts (headers counted, or not)
针对论文中的每一个数字（数据集规模、指标值、百分比、计数）：
  1. 提取 .tex文件中的数字及其上下文
  2. 追溯其来源：代码输出、结果文件、日志或跟踪系统
  3. 验证数值是否完全匹配（注意四舍五入、百分比与小数的区别）
  4. 标记任何无法追溯到来源的数字
模板：
| Paper claim | Location (.tex) | Source file/code | Source value | Match? |
|-------------|-----------------|-----------------|-------------|--------|
| "13,999 frames" | abstract L3 | len(glob(labels/*.json)) | ? | ? |
| "4.2% improvement" | Table 2 | eval_results.json | ? | ? |
常见数值错误：
  • 四舍五入不一致（文本中为3.14，表格中为3.1415）
  • 重新运行实验后未更新的早期实验旧数据
  • 百分比与绝对值混淆
  • 数据集计数的差一错误（是否统计了表头）

2. Terminology Consistency Audit

2. 术语一致性审计

  1. Extract all defined terms from the methods section
  2. Search for each term across ALL sections
  3. Flag any inconsistent usage:
    • Same concept, different names (e.g., "tag head" vs "classification head")
    • Same name, different meanings across sections
    • Defined but never used, or used but never defined
  1. 提取方法部分中所有定义的术语
  2. 搜索每个术语在所有章节中的使用情况
  3. 标记任何不一致的用法：
    • 同一概念使用不同名称（例如“tag head”与“classification head”）
    • 同一名称在不同章节中含义不同
    • 已定义但从未使用，或已使用但从未定义

3. Code-Paper Alignment

3. 代码-论文一致性检查

For each method described in the paper:
  1. Find the corresponding code (function, class, module)
  2. Compare the paper's description with the actual implementation
  3. Check specifically:
    • Algorithm steps match code flow
    • Hyperparameters in text match config/code defaults
    • Architecture descriptions match model code
    • Loss functions in equations match loss code
    • Training procedures match training scripts
Common mismatches:
  • Paper describes an idealized version, code has edge cases not mentioned
  • Hyperparameters changed during development but paper not updated
  • Paper describes a method that was later modified or removed from code
针对论文中描述的每一种方法：
  1. 找到对应的代码（函数、类、模块）
  2. 对比论文描述与实际实现
  3. 重点检查：
    • 算法步骤与代码流程匹配
    • 文本中的超参数与配置/代码默认值匹配
    • 架构描述与模型代码匹配
    • 公式中的损失函数与损失代码匹配
    • 训练流程与训练脚本匹配
常见不匹配情况：
  • 论文描述的是理想化版本，代码包含未提及的边缘情况
  • 开发过程中超参数已更改，但论文未更新
  • 论文描述的方法后来被修改或从代码中移除

4. Formula-Code Verification

4. 公式-代码验证

For each equation in the paper:
  1. Identify the equation and its variables
  2. Find the code that implements it
  3. Map each mathematical operation to its code equivalent
  4. Verify:
    • Summation bounds match loop bounds
    • Division operations handle edge cases
    • Normalization factors match
    • Gradient flow matches (detach, no_grad)
    • Reduction operations (mean vs sum) match
针对论文中的每一个公式：
  1. 识别公式及其变量
  2. 找到实现该公式的代码
  3. 映射每个数学运算到对应的代码实现
  4. 验证：
    • 求和范围与循环范围匹配
    • 除法运算处理了边缘情况
    • 归一化因子匹配
    • 梯度流匹配（detach、no_grad）
    • 归约运算（均值vs求和）匹配

5. Citation Fact-Checking Protocol

5. 引用事实核查流程

For each citation in the paper:
Step 1: Extract the claim and the cited paper Step 2: Verify BibTeX metadata against DBLP:
  • Author names (exact spelling, correct order)
  • Paper title (exact, from published version not preprint)
  • Venue and year (confirmed against actual publication)
Step 3: For cited claims with specific numbers:
  • Locate the exact table/figure in the cited paper
  • Verify the number matches what the citing paper states
  • If the number cannot be confirmed, suggest qualitative language instead
Step 4: Check for common citation errors:
  • Citing preprint when published version exists
  • Wrong year (submission vs publication)
  • Author name misspellings
  • Citing for a claim the paper doesn't actually make
针对论文中的每一处引用：
步骤1：提取声明及被引用的论文 步骤2：验证BibTeX元数据与DBLP是否一致：
  • 作者姓名（拼写准确、顺序正确）
  • 论文标题（与已发表版本完全一致，而非预印本）
  • 发表场所和年份（与实际出版物确认）
步骤3：针对包含具体数字的引用声明：
  • 定位被引用论文中的对应表格/图表
  • 验证数字与引用论文中的表述是否一致
  • 如果无法确认数字，建议改用定性语言
步骤4：检查常见引用错误：
  • 已有发表版本却引用预印本
  • 年份错误（提交年份vs发表年份）
  • 作者姓名拼写错误
  • 引用的内容并非被引用论文实际提出的声明

Verification Process

验证流程

  1. Read the full paper (or specified sections)
  2. Build the verification table for each dimension
  3. For each entry, read the source and verify
  4. Produce a prioritized issue list:
    • HIGH: Incorrect numbers, wrong claims, missing citations
    • MEDIUM: Terminology inconsistencies, stale but close numbers
    • LOW: Minor formatting, optional improvements
  1. 阅读完整论文（或指定章节）
  2. 为每个维度构建验证表格
  3. 针对每一项，查阅来源并进行验证
  4. 生成优先级问题列表：
    • 高优先级：错误数字、错误声明、缺失引用
    • 中优先级：术语不一致、过时但接近正确的数字
    • 低优先级：轻微格式问题、可选改进项

Output Format

输出格式

Produce a structured verification report:
  1. Summary: X issues found (Y high, Z medium, W low)
  2. Numerical audit table: each number with source and match status
  3. Terminology issues: inconsistent terms with locations
  4. Code-paper mismatches: description vs implementation gaps
  5. Citation issues: metadata errors and unverified claims
  6. Suggested fixes: specific text replacements for each issue
生成结构化的验证报告：
  1. 摘要：共发现X个问题（Y个高优先级、Z个中优先级、W个低优先级）
  2. 数值审计表格：每个数字的来源及匹配状态
  3. 术语问题：不一致术语及其位置
  4. 代码-论文不匹配：描述与实现的差异
  5. 引用问题：元数据错误及未验证的声明
  6. 建议修复方案：针对每个问题的具体文本替换建议