Research Review via Codex MCP (xhigh reasoning)

Constants

• REVIEWER_MODEL =

  gpt-5.4

  — Model used via Codex MCP. Must be an OpenAI model (e.g.,

  gpt-5.4

  ,

  o3

  ,

  gpt-4o

  )

Context: $ARGUMENTS

Prerequisites

• Codex MCP Server configured in Claude Code:
  bash

  claude mcp add codex -s user -- codex mcp-server

• This gives Claude Code access to

  mcp__codex__codex

  and

  mcp__codex__codex-reply

  tools

Workflow

Step 1: Gather Research Context

1. Read project narrative documents (e.g., STORY.md, README.md, paper drafts)
2. Read any memory/notes files for key findings and experiment history
3. Identify: core claims, methodology, key results, known weaknesses

Step 2: Initial Review (Round 1)

mcp__codex__codex:
  config: {"model_reasoning_effort": "xhigh"}
  prompt: |
    [Full research context + specific questions]
    Please act as a senior ML reviewer (NeurIPS/ICML level). Identify:
    1. Logical gaps or unjustified claims
    2. Missing experiments that would strengthen the story
    3. Narrative weaknesses
    4. Whether the contribution is sufficient for a top venue
    Please be brutally honest.

Step 3: Iterative Dialogue (Rounds 2-N)

mcp__codex__codex-reply

threadId

1. Respond to criticisms with evidence/counterarguments
2. Ask targeted follow-ups on the most actionable points
3. Request specific deliverables: experiment designs, paper outlines, claims matrices

• "If we reframe X as Y, does that change your assessment?"
• "What's the minimum experiment to satisfy concern Z?"
• "Please design the minimal additional experiment package (highest acceptance lift per GPU week)"
• "Please write a mock NeurIPS/ICML review with scores"
• "Give me a results-to-claims matrix for possible experimental outcomes"

Step 4: Convergence

• Both sides agree on the core claims and their evidence requirements
• A concrete experiment plan is established
• The narrative structure is settled

Step 5: Document Everything

• Round-by-round summary of criticisms and responses
• Final consensus on claims, narrative, and experiments
• Claims matrix (what claims are allowed under each possible outcome)
• Prioritized TODO list with estimated compute costs
• Paper outline if discussed

Key Rules

• ALWAYS use

  config: {"model_reasoning_effort": "xhigh"}

  for reviews
• Send comprehensive context in Round 1 — the external model cannot read your files
• Be honest about weaknesses — hiding them leads to worse feedback
• Push back on criticisms you disagree with, but accept valid ones
• Focus on ACTIONABLE feedback — "what experiment would fix this?"
• Document the threadId for potential future resumption
• The review document should be self-contained (readable without the conversation)

Prompt Templates

For initial review:

For experiment design:

For paper structure:

For claims matrix:

For mock review: