Back to Details

fact-check

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Fact-Check Skill

事实核查Skill

Systematic verification of claims in generated content. Designed to catch hallucinations, confabulations, and unsupported assertions.
对生成内容中的主张进行系统性验证。旨在识别hallucinations、虚构内容和无依据的断言。

Why Separate Passes Matter

为何需要单独步骤

The Fundamental Problem: LLMs generate plausible-sounding content by predicting what should come next. This same mechanism produces hallucinations—confident statements that feel true but aren't. An LLM in generation mode cannot reliably catch its own hallucinations because:
  1. Attention is on generation, not verification
  2. Coherence pressure makes false claims feel correct in context
  3. Same weights that produced the error will confirm it
  4. No external grounding to contradict the confabulation
The Solution: Verification must be a separate cognitive pass with:
  • Fresh attention focused solely on each claim
  • Explicit source checking (not memory/training data)
  • Adversarial stance toward the content
  • External grounding where possible
核心问题： LLMs通过预测后续内容生成看似合理的内容。正是这种机制会产生hallucinations——那些听起来真实但实际并非如此的笃定陈述。处于生成模式的LLM无法可靠地识别自身产生的hallucinations，原因如下：
  1. 注意力集中在生成上，而非验证
  2. 连贯性压力使错误主张在语境中看起来合理
  3. 产生错误的相同权重参数会反过来确认该错误
  4. 没有外部依据来反驳虚构内容
解决方案： 验证必须作为一个独立的认知步骤，具备以下特点：
  • 全新的注意力仅集中在每个主张上
  • 明确的来源核查（而非依赖记忆/训练数据）
  • 对内容采取质疑性立场
  • 尽可能使用外部依据

Diagnostic States

诊断状态

F1: No Verification Pass

F1：未执行验证步骤

Symptoms: Content generated and delivered without any fact-checking. Risk: Hallucinations pass through undetected. Intervention: Run verification pass before delivery. Extract claims, check each against sources.
症状： 内容生成后直接交付，未进行任何事实核查。 风险： Hallucinations未被识别就直接交付。 干预措施： 在交付前执行验证步骤。提取所有主张，逐一对照来源核查。

F2: Self-Verification (Invalid)

F2：自我验证（无效）

Symptoms: Same pass asked to "check your facts" while generating. Risk: False confidence—errors confirmed by same process that created them. Intervention: Complete generation first, then run separate verification pass with explicit source requirements.
症状： 在生成内容的同时要求模型“检查事实”。 风险： 虚假自信——产生错误的同一过程会反过来确认该错误。 干预措施： 先完成内容生成，再执行独立的验证步骤，并明确要求使用来源。

F3: Memory-Based Verification (Unreliable)

F3：基于记忆的验证（不可靠）

Symptoms: Claims checked against "what I know" without external sources. Risk: Hallucinations verified by hallucinated knowledge. Intervention: Require explicit source citation for each verified claim. If no source available, mark as unverified.
症状： 仅对照“自身所知”核查主张，未使用外部来源。 风险： 依靠幻觉知识来验证hallucinations。 干预措施： 要求每个已验证的主张都有明确的来源引用。如果没有可用来源，标记为未验证。

F4: Selective Verification

F4：选择性验证

Symptoms: Only some claims checked; others assumed correct. Risk: Unchecked claims may contain errors. Intervention: Systematic extraction of ALL verifiable claims. Check each, or explicitly mark unchecked items.
症状： 仅核查部分主张，其余主张默认正确。 风险： 未被核查的主张可能包含错误。 干预措施： 系统性提取所有可验证的主张。逐一核查，或明确标记未核查的内容。

F5: Verification Complete

F5：验证完成

Symptoms: All claims extracted, each checked against sources, confidence levels assigned. Indicators: Source citations present, unverified claims marked, confidence explicit.
症状： 所有主张已提取，每个主张都对照来源进行了核查，并分配了置信度。 指标： 存在来源引用，未验证的主张已标记，置信度明确。

The Verification Process

验证流程

Phase 1: Claim Extraction

阶段1：主张提取

Extract every verifiable statement from the content.
Claim types to extract:
  • Factual assertions ("X is Y", "X causes Y")
  • Statistics and numbers ("40% of...", "in 2023...")
  • Attributions ("According to X...", "Research shows...")
  • Definitions ("X means...", "X is defined as...")
  • Historical claims ("X happened in...", "X was founded by...")
  • Causal claims ("X leads to Y", "X prevents Y")
  • Comparative claims ("X is better than Y", "X is the largest...")
What to skip:
  • Opinions clearly marked as such
  • Hypotheticals and speculation (if labeled)
  • Logical deductions from stated premises
  • Direct quotes (verify attribution, not content)
从内容中提取所有可验证的陈述。
需提取的主张类型：
  • 事实断言（“X是Y”，“X导致Y”）
  • 统计数据和数字（“40%的……”，“2023年……”）
  • 归属信息（“根据X……”，“研究表明……”）
  • 定义（“X指的是……”，“X被定义为……”）
  • 历史主张（“X发生在……”，“X由……创立”）
  • 因果主张（“X导致Y”，“X预防Y”）
  • 比较主张（“X优于Y”，“X是最大的……”）
可跳过的内容：
  • 明确标记为观点的内容
  • 假设和推测（如果已标注）
  • 基于既定前提的逻辑推导
  • 直接引用（仅验证归属信息，不验证内容本身）

Phase 2: Claim Categorization

阶段2：主张分类

Categorize each claim by verifiability:
CategoryDescriptionVerification Strategy
Verifiable-HardNumbers, dates, names, quotesMust match source exactly
Verifiable-SoftGeneral facts, processes, mechanismsSource should substantially support
Attribution"X said...", "According to..."Verify source exists and said something similar
InferenceConclusions drawn from evidenceVerify premises, assess reasoning
Opinion-as-FactSubjective claim stated as objectiveFlag for rewording or qualification
根据可验证性对每个主张进行分类：
类别描述验证策略
强可验证数字、日期、姓名、引用必须与来源完全匹配
弱可验证一般事实、流程、机制来源应提供实质性支持
归属信息“X表示……”，“根据……”验证来源存在且发表过类似内容
推论从证据中得出的结论验证前提，评估推理过程
伪装成事实的观点主观主张被表述为客观事实标记为需要重新措辞或限定

Phase 3: Source Verification

阶段3：来源核查

For each claim, attempt verification:
markdown
undefined
对每个主张尝试进行核查：
markdown
undefined

Claim Verification Log

Claim Verification Log

Claim 1: "[exact claim text]"

Claim 1: "[exact claim text]"

  • Category: [Verifiable-Hard/Soft/Attribution/Inference]
  • Source checked: [specific source]
  • Finding: [Confirmed/Partially supported/Not found/Contradicted]
  • Confidence: [High/Medium/Low]
  • Notes: [discrepancies, qualifications needed]
  • Category: [Verifiable-Hard/Soft/Attribution/Inference]
  • Source checked: [specific source]
  • Finding: [Confirmed/Partially supported/Not found/Contradicted]
  • Confidence: [High/Medium/Low]
  • Notes: [discrepancies, qualifications needed]

Claim 2: ...

Claim 2: ...


**Verification outcomes:**

| Outcome | Meaning | Action |
|---------|---------|--------|
| **Confirmed** | Source explicitly supports claim | Keep, cite source |
| **Partially supported** | Source supports part, not all | Qualify or narrow claim |
| **Not found** | No source located | Mark unverified, consider removing |
| **Contradicted** | Source says opposite | Remove or correct |
| **Outdated** | Source is dated; current state may differ | Update or add recency caveat |

**核查结果：**

| 结果 | 含义 | 操作 |
|---------|---------|--------|
| **已确认** | 来源明确支持该主张 | 保留并引用来源 |
| **部分支持** | 来源仅支持部分内容，而非全部 | 对主张进行限定或缩小范围 |
| **未找到来源** | 未定位到相关来源 | 标记为未验证，考虑移除 |
| **存在矛盾** | 来源内容与主张相反 | 移除或修正主张 |
| **已过时** | 来源数据过时，当前情况可能已变化 | 更新内容或添加时效性说明 |

Phase 4: Confidence Assignment

阶段4：置信度分配

Assign overall confidence to the content:
LevelCriteria
HighAll key claims verified; no contradictions found
MediumMost claims verified; some unverified but plausible
LowSignificant claims unverified; some corrections needed
UnreliableMultiple contradictions found; major revision needed
为内容分配整体置信度：
等级标准
高所有关键主张已验证，未发现矛盾
中大多数主张已验证，部分未验证但看似合理
低重要主张未验证，部分内容需要修正
不可靠发现多处矛盾，需要大幅修订

Hallucination Patterns

Hallucination模式

Common hallucination types to watch for:
需要关注的常见hallucinations类型：

1. Plausible Fabrication

1. 看似合理的编造

Pattern: Specific details that sound right but don't exist. Examples: Fake paper citations, non-existent statistics, invented quotes. Detection: Verify specific claims against primary sources.
模式： 听起来合理但实际不存在的具体细节。 示例： 虚假的论文引用、不存在的统计数据、编造的引用。 检测方法： 对照原始来源验证具体主张。

2. Confident Extrapolation

2. 笃定的推断

Pattern: Reasonable inference stated as established fact. Examples: "Studies show..." (no specific study), "Experts agree..." (no citation). Detection: Require specific source for any claim of external support.
模式： 合理的推论被表述为既定事实。 示例： “研究表明……”（无具体研究），“专家一致认为……”（无引用）。 检测方法： 任何声称有外部支持的主张都需要提供具体来源。

3. Temporal Confusion

3. 时间混淆

Pattern: Mixing information from different time periods. Examples: Old statistics presented as current, defunct organizations described as active. Detection: Check dates on sources, verify current status.
模式： 混合不同时间段的信息。 示例： 将旧统计数据作为当前数据呈现，已解散的组织被描述为活跃状态。 检测方法： 检查来源的日期，验证当前状态。

4. Attribution Drift

4. 归属偏差

Pattern: Correct information attributed to wrong source. Examples: Quote assigned to wrong person, finding attributed to wrong study. Detection: Verify attribution specifically, not just content.
模式： 正确的信息被归属于错误的来源。 示例： 引用被分配给错误的人，研究结果被归属于错误的研究。 检测方法： 专门验证归属信息，而非仅验证内容。

5. Amalgamation

5. 信息融合

Pattern: Combining details from multiple sources into one fictional source. Examples: Invented study that combines real findings from separate papers. Detection: Verify the specific source exists and contains all attributed claims.
模式： 将多个来源的细节合并成一个虚构的来源。 示例： 编造的研究，整合了来自不同论文的真实发现。 检测方法： 验证具体来源是否存在，且包含所有被归属的主张。

6. Precision Inflation

6. 精度膨胀

Pattern: Adding false precision to vague knowledge. Examples: "Approximately 47.3%" when only "about half" is supported. Detection: Check if source actually provides that level of precision.
模式： 为模糊的知识添加虚假的精度。 示例： 当仅支持“约一半”时，表述为“约47.3%”。 检测方法： 检查来源是否确实提供了该精度的数据。

Verification Checklist

验证检查清单

Before releasing fact-checked content:
  • Claims extracted? All verifiable statements identified
  • Sources checked? Each claim verified against external source
  • Specific, not memory? Verification used actual sources, not LLM training data
  • Contradictions flagged? Conflicts between claims and sources noted
  • Unverified marked? Claims without sources explicitly identified
  • Confidence stated? Overall reliability level communicated
  • Separate pass? Verification done after generation, not during
在发布经过事实核查的内容前：
  • 主张已提取？ 所有可验证的陈述已识别
  • 来源已核查？ 每个主张都已对照外部来源验证
  • 使用的是具体来源而非记忆？** 验证使用的是实际来源，而非LLM的训练数据
  • 矛盾已标记？ 主张与来源之间的冲突已记录
  • 未验证内容已标记？ 无来源的主张已明确识别
  • 置信度已说明？ 已传达整体可靠性等级
  • 是否为独立步骤？ 验证在生成完成后执行，而非生成过程中

Integration with Research Skill

与研究Skill的集成

Research PhaseFact-Check Role
During researchVerify claims in sources themselves
After synthesisVerify that synthesis accurately represents sources
Before deliveryFinal pass to catch hallucinations in output
Handoff pattern:
  1. Research skill gathers and synthesizes information
  2. Content is generated based on research
  3. Fact-check skill runs as separate pass
  4. Corrections made, confidence assigned
  5. Output delivered with verification status
研究阶段事实核查角色
研究期间验证来源本身的主张
合成后验证合成内容是否准确代表来源
交付前最终步骤，识别输出中的hallucinations
交接模式：
  1. 研究Skill收集并综合信息
  2. 基于研究结果生成内容
  3. 事实核查Skill作为独立步骤运行
  4. 进行修正，分配置信度
  5. 交付带有验证状态的输出

Operational Constraints

操作限制

What This Skill Cannot Do

本Skill无法完成的任务

  1. Verify during generation — Must be separate pass
  2. Catch all hallucinations — Some may slip through
  3. Verify without sources — No sources = unverified, not "verified by knowledge"
  4. Replace domain expertise — Can check sources exist, not evaluate quality
  1. 生成过程中验证 — 必须作为独立步骤
  2. 识别所有hallucinations — 部分可能未被发现
  3. 无来源时验证 — 无来源 = 未验证，而非“通过知识验证”
  4. 替代领域专业知识 — 可以检查来源是否存在，但无法评估来源质量

When Verification Is Most Critical

验证最关键的场景

ContextVerification Level
Published contentFull verification required
Decision supportKey claims must be verified
Educational contentHigh accuracy expected
Casual conversationLight verification acceptable
Creative fictionN/A (different standards)
场景验证等级
发布内容需要完整验证
决策支持关键主张必须验证
教育内容要求高准确性
随意对话可接受轻量验证
创意虚构内容不适用（标准不同）

Anti-Patterns

反模式

PatternProblemFix
"I'm confident"Confidence ≠ accuracyRequire source citation
"To the best of my knowledge"Memory is unreliableCheck external source
"Generally speaking"Vagueness hides uncertaintyBe specific or mark unverified
"Research shows"Which research?Cite specific source
Verify-while-generatingSame pass can't catch own errorsSeparate passes mandatory
Check one, assume restPartial verificationCheck all or mark unchecked
模式问题修复方案
“我很确定”自信 ≠ 准确性要求提供来源引用
“据我所知”记忆不可靠检查外部来源
“一般来说”模糊性隐藏不确定性具体化或标记为未验证
“研究表明”哪项研究？引用具体来源
生成时验证同一步骤无法识别自身错误必须使用独立步骤
核查部分，假设其余正确验证不完整全部核查或标记未核查内容

Output Format

输出格式

When delivering fact-checked content:
markdown
undefined
交付经过事实核查的内容时：
markdown
undefined

[Content Title]

[Content Title]

[Content body with claims]
[Content body with claims]

Verification Status

Verification Status

Overall Confidence: [High/Medium/Low]
Verified Claims:
  • [Claim 1] — Source: [citation]
  • [Claim 2] — Source: [citation]
Unverified Claims:
  • [Claim 3] — No source found; treat as uncertain
Corrections Made:
  • [Original claim] → [Corrected claim] (Source: [citation])
Caveats:
  • [Any limitations or qualifications]
undefined
Overall Confidence: [High/Medium/Low]
Verified Claims:
  • [Claim 1] — Source: [citation]
  • [Claim 2] — Source: [citation]
Unverified Claims:
  • [Claim 3] — No source found; treat as uncertain
Corrections Made:
  • [Original claim] → [Corrected claim] (Source: [citation])
Caveats:
  • [Any limitations or qualifications]
undefined

Output Persistence

输出持久化

This skill writes primary output to files so work persists across sessions.
本Skill会将主要输出写入文件，以便跨会话保留工作成果。

Output Discovery

输出查找

Before doing any other work:
  1. Check for 
    context/output-config.md
    in the project
  2. If found, look for this skill's entry
  3. If not found or no entry for this skill, ask the user first:
    • "Where should I save output from this fact-check session?"
    • Suggest: 
      explorations/fact-check/
      or a sensible location for this project
  4. Store the user's preference:
    • In 
      context/output-config.md
      if context network exists
    • In 
      .fact-check-output.md
      at project root otherwise
在执行任何其他工作前：
  1. 检查项目中的
    context/output-config.md
  2. 如果找到，查找本Skill的条目
  3. 如果未找到或无本Skill的条目，先询问用户：
    • “我应该将本次事实核查会话的输出保存到哪里？”
    • 建议：
      explorations/fact-check/
      或适合本项目的合理位置
  4. 存储用户的偏好：
    • 如果存在上下文网络，保存到
      context/output-config.md
    • 否则保存到项目根目录的
      .fact-check-output.md

Primary Output

主要输出

For this skill, persist:
  • Claims extracted - all verifiable statements identified
  • Verification results - each claim with source and status
  • Confidence assessment - overall content reliability
  • Corrections made - any changes from original
对于本Skill，需持久化以下内容：
  • 提取的主张 - 所有已识别的可验证陈述
  • 验证结果 - 每个主张的来源和状态
  • 置信度评估 - 内容的整体可靠性
  • 已做修正 - 与原始内容的所有更改

Conversation vs. File

对话 vs 文件

Goes to FileStays in Conversation
Verification status reportDiscussion of sources
Claim-by-claim resultsClarifying questions
Confidence assessmentVerification process
Corrections and caveatsReal-time feedback
存入文件保留在对话中
验证状态报告关于来源的讨论
主张逐一核查结果澄清问题
置信度评估验证流程
修正和说明实时反馈

File Naming

文件命名

Pattern: 
{content-name}-factcheck-{date}.md
Example: 
research-synthesis-factcheck-2025-01-15.md
格式：
{content-name}-factcheck-{date}.md
示例：
research-synthesis-factcheck-2025-01-15.md

Source Framework

来源框架

This skill extends the research cluster with post-generation verification. Distinct from research (which gathers information) and operates as quality control on output.
Related: 
skills/research/SKILL.md
(pre-generation), 
references/doppelganger/
(truth hierarchies)
本Skill扩展了研究集群，增加了生成后的验证环节。与研究（收集信息）不同，本Skill作为输出的质量控制环节。
相关内容：
skills/research/SKILL.md
（生成前），
references/doppelganger/
（真实层级）