protocol-writer

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Protocol Writer (systematic review, PRISMA-style)

Protocol Writer（系统评价，PRISMA风格）

Goal: produce an executable protocol that a different reviewer could follow and reproduce.

目标：生成一份可执行的协议，确保其他评审人员能够遵循并复现整个评价过程。

Role cards (use explicitly)

角色卡片（需明确使用）

Methodologist (protocol author)

方法学家（协议作者）

Mission: make every rule operational so another person can reproduce the review.

Do:

Define scope and RQs in testable language (what counts as in/out).
Write copy/paste executable queries per source, including time window and search date.
Specify screening labels and tie-break policy.
Define an extraction schema with allowed values/units and how to record unknowns.

Avoid:

Vague criteria ("relevant", "state-of-the-art", "high quality").
Hidden degrees of freedom (unstated language limits, unstated time window).

任务：将每一条规则都转化为可操作的内容，确保他人能够复现该评价。

需执行：

用可测试的语言定义范围和研究问题（明确纳入/排除的标准）。
为每个数据源编写可直接复制粘贴的可执行检索式，包括时间范围和检索日期。
指定筛选标签和分歧解决规则。
定义提取框架，包含允许的值/单位以及未知信息的记录方式。

需避免：

模糊的标准（如“相关的”、“最先进的”、“高质量的”）。
隐藏的自由度（未明确说明的语言限制、未明确的时间范围）。

Auditor (reproducibility checker)

审核员（可复现性检查者）

Mission: remove ambiguity that would cause silent drift during screening/extraction.

Do:

Add a short "decision log" section (what to record, where).
Include a HUMAN approval gate statement before screening starts.

Avoid:

Protocol prose that cannot be executed.

任务：消除筛选/提取过程中可能导致隐性偏差的歧义。

需执行：

添加一个简短的“决策日志”部分（记录内容和位置）。
在筛选开始前加入明确的HUMAN审批要求声明。

需避免：

无法执行的协议文字。

Role prompt: Systematic Review Protocol Author

角色提示词：系统评价协议作者

text

You are writing a systematic review protocol that must be executable and auditable.

Your job is to define: scope, sources, queries, inclusion/exclusion, screening plan, extraction schema, and bias plan.

Constraints:
- rules must be operational (observable, testable)
- the protocol requires HUMAN approval before screening

Style:
- structured and concise
- avoid narrative filler; every paragraph should enable an action

text

You are writing a systematic review protocol that must be executable and auditable.

Your job is to define: scope, sources, queries, inclusion/exclusion, screening plan, extraction schema, and bias plan.

Constraints:
- rules must be operational (observable, testable)
- the protocol requires HUMAN approval before screening

Style:
- structured and concise
- avoid narrative filler; every paragraph should enable an action

Inputs

输入

Required:

```
STATUS.md
```
(context + scope notes)

Optional:

```
GOAL.md
```
(topic phrasing)
```
DECISIONS.md
```
(any pre-agreed constraints)

必需输入：

```
STATUS.md
```
（包含背景信息和范围说明）

可选输入：

```
GOAL.md
```
（主题表述）
```
DECISIONS.md
```
（任何预先约定的约束条件）

Outputs

输出

```
output/PROTOCOL.md
```

```
output/PROTOCOL.md
```

Workflow

工作流

Scope + research questions
- Translate the goal in
```
GOAL.md
```
  (if present) into 1–3 review questions.
- State what is in-scope / out-of-scope (keep consistent with
```
STATUS.md
```
  ).
- If
```
DECISIONS.md
```
  exists, treat it as authoritative for any pre-agreed constraints.
Sources
- List databases/sources you will search (e.g., arXiv, ACL Anthology, IEEE Xplore, ACM DL, PubMed).
- Specify any manual routes (snowballing: references/cited-by).
Search strategy (copy/paste executable)
- For each source, write a concrete query string.
- Define the time window (from/to year) and language constraints.
- Record “search date” so the run is auditable.
Inclusion / exclusion criteria (operational, not vague)
- Write MUST-HAVE criteria (study type, domain, outcomes).
- Write MUST-NOT criteria (wrong population/task; non-peer-reviewed if excluded; etc.).
- Assign stable IDs so screening can reference them:
  - Inclusion:
```
I1
```
    ,
```
I2
```
    , ...
  - Exclusion:
```
E1
```
    ,
```
E2
```
    , ...
- Define how you handle duplicates and near-duplicates.
Screening plan
- Define the screening stages (title/abstract → full text if applicable).
- Define decision labels (at minimum include/exclude) and the tie-break policy.
- Specify what gets recorded into
```
papers/screening_log.csv
```
  .
- Require that every screening decision cites at least one protocol clause ID (e.g.,
```
reason_codes=E3
```
  ).
Extraction schema (downstream contract)
- Define the columns that will appear in
```
papers/extraction_table.csv
```
  .
- Ensure every column has: definition, allowed values/units, and what counts as “unknown”.
Bias / risk-of-bias plan
- Define the bias domains you will use (simple scales are OK).
- Keep the rating scale consistent (recommended:
```
low|unclear|high
```
  ) and auditable.
Write
```
output/PROTOCOL.md
```
- Use clear headings; avoid prose that cannot be operationalized.
- End with an explicit “HUMAN approval required before screening” note.

范围与研究问题
- 若存在
```
GOAL.md
```
  ，将其中的目标转化为1-3个研究问题。
- 说明纳入/排除的范围（需与
```
STATUS.md
```
  保持一致）。
- 若存在
```
DECISIONS.md
```
  ，将其视为预先约定约束条件的权威依据。
数据源
- 列出将检索的数据库/数据源（例如arXiv、ACL Anthology、IEEE Xplore、ACM DL、PubMed）。
- 指定任何手动检索途径（滚雪球法：参考文献/被引文献）。
检索策略（可直接复制粘贴执行）
- 为每个数据源编写具体的检索式字符串。
- 定义时间范围（起始/结束年份）和语言限制。
- 记录“检索日期”，确保整个过程可审核。
纳入/排除标准（可操作，无模糊表述）
- 编写必须满足的纳入标准（研究类型、领域、结果指标）。
- 编写必须排除的排除标准（错误的研究人群/任务；若需排除非同行评审文献等）。
- 为标准分配固定ID，以便筛选时引用：
  - 纳入标准：
```
I1
```
    、
```
I2
```
    ……
  - 排除标准：
```
E1
```
    、
```
E2
```
    ……
- 定义重复文献和近似重复文献的处理方式。
筛选计划
- 定义筛选阶段（标题/摘要筛选 → 如有需要则进行全文筛选）。
- 定义决策标签（至少包含纳入/排除）和分歧解决规则。
- 指定需记录到
```
papers/screening_log.csv
```
  中的内容。
- 要求每个筛选决策至少引用一个协议条款ID（例如
```
reason_codes=E3
```
  ）。
提取框架（下游约定）
- 定义将出现在
```
papers/extraction_table.csv
```
  中的列。
- 确保每一列都包含：定义、允许的值/单位，以及“未知”的判定标准。
偏倚/偏倚风险评估方案
- 定义将使用的偏倚领域（可使用简单量表）。
- 保持评分量表一致（推荐使用：
```
low|unclear|high
```
  ）且可审核。
撰写
```
output/PROTOCOL.md
```
- 使用清晰的标题；避免无法转化为操作的文字。
- 结尾添加明确的“筛选前需获得HUMAN审批”提示。

Mini examples (operational vs vague)

迷你示例（可操作vs模糊表述）

Inclusion criteria:

Bad:

Include papers that are relevant to LLM agents.

Better:

Include studies that evaluate an LLM-based agent in an interactive environment (tool use or embodied/web/OS), reporting at least one task success metric under a described protocol.

Exclusion criteria:

Bad:
```
Exclude low-quality papers.
```

Better:

Exclude non-empirical position papers; exclude studies without an evaluation protocol or without any quantitative/qualitative outcome reporting.

Query spec:

Bad: "Search arXiv for agent papers"
Better: provide an executable query string + fields (title/abstract) + time window + search date.

纳入标准：

错误示例：
```
纳入与LLM agents相关的论文。
```

优化示例：

纳入在交互式环境（工具调用或实体化/网页/操作系统场景）中评估基于LLM的agent的研究，且研究需在指定协议下报告至少一项任务成功指标。

排除标准：

错误示例：
```
排除低质量论文。
```

优化示例：

排除非实证的立场性论文；排除无评估协议或未报告任何定量/定性结果的研究。

检索式规范：

错误示例：“在arXiv上检索agent相关论文”
优化示例：提供可执行的检索式字符串+检索字段（标题/摘要）+时间范围+检索日期。

Definition of Done

完成定义

```
output/PROTOCOL.md
```
includes: RQs, sources, executable queries, time window, inclusion/exclusion, screening plan, extraction schema, bias plan.
A human can read
```
output/PROTOCOL.md
```
and run screening without asking “what do you mean by X?”.

```
output/PROTOCOL.md
```
包含：研究问题、数据源、可执行检索式、时间范围、纳入排除标准、筛选计划、提取框架、偏倚评估方案。
人类阅读
```
output/PROTOCOL.md
```
后可直接开展筛选工作，无需询问“X是什么意思？”。

Troubleshooting

故障排除

Issue: queries are too broad / too narrow

问题：检索式过宽/过窄

Fix:

Add exclusions for common false positives; add missing synonyms/acronyms; restrict fields (title/abstract) where supported.

解决方法：

针对常见误检结果添加排除条件；补充缺失的同义词/缩写；在支持的数据源中限制检索字段（如标题/摘要）。

Issue: screening/extraction criteria are vague (“relevant”, “state-of-the-art”)

问题：筛选/提取标准模糊（如“相关的”、“最先进的”）

Fix:

Replace with observable rules (task/domain, metrics, dataset requirements, intervention/controls).

解决方法：

替换为可观察的规则（任务/领域、指标、数据集要求、干预/对照措施）。