PR Review Report Creation Procedure Manual
Reviews feature/bugfix/refactoring PRs and creates review reports. In addition to supporting regular code reviews, it also serves as a self-review tool for implementers to review their own PRs. The goal is to produce a sound pull request that can be presented to human reviewers.
Out of Scope
- Library update PRs (from dependabot/renovatebot, etc.) are covered by the skill and are out of scope for this skill
Procedure
Step 0: Retrieve PR Information and Confirm Report Output Destination
If a PR number or URL is specified as an argument, that PR will be the target.
If no specification is provided, the PR associated with the current branch will be automatically detected.
In either case, retrieve PR information using the following command:
gh pr view {PR number or URL} --json number,title,body,url,author,comments,headRefName
For automatic detection, omit
.
If no PR is found, notify the user and terminate.
The PR title, PR number, and branch name will be used in the report header. Enter the current date and time (YYYY-MM-DD HH:mm:ss) in "Reviewed at", and your own Agent name in "Reviewer".
Retrieve the following information:
- PR body (description)
- PR comments: comments from
- Inline review comments:
gh api repos/{owner}/{repo}/pulls/{number}/comments --paginate
- PR reviews:
gh api repos/{owner}/{repo}/pulls/{number}/reviews --paginate
- Diff:
Confirm Report Output Destination
Before starting the review work, confirm the report output destination using the AskUserQuestion tool:
- Chat and pull request (recommended): Output the review report to the conversation, and post the same content as a comment on the PR
- Chat: Output the review report only to the conversation
If the user has explicitly specified the output destination, skip this confirmation and follow the specification.
Step 1: Load Conversation Context
Search for conversation context in the following order:
1-1. Check PR Comment Section
Check PR comments for any comment containing the title "Conversation Context".
If found, use its content as the conversation context.
1-2. Check .dev/contexts/
Sanitize the branch name (replace
with
), and look for
.dev/contexts/{sanitized-branch-name}.md
.
If found, load it using the Read tool.
1-3. If Neither Is Found
Confirm the following using the AskUserQuestion tool:
- Continue without conversation context: Skip Step 6 (Check for Oversights)
- Abort: Ask the user to prepare the conversation context
Step 2: Quality Assessment of Pull Request Description
Perform this step before reading the code. This is to prevent overlooking structural issues in the description due to being distracted by code consistency checks.
The implementer may not have fully thought through their work. They might have written the PR description without fully understanding it, or just pasted AI-generated content as-is.
In this step, only the description field is read to evaluate whether a reviewer can "judge the validity of the changes" by reading this description.
PR Description Checklist
Be sure to evaluate all four items below and note the results. Proceed to the next step only after evaluating all items:
- Is the pre-change behavior/problem described?: Reviewers need to know the state before the change to judge its validity. Only describing "what was done" leaves the question: "Wait, maybe the original behavior was more correct?"
- Is it written in pairs like "Problem → Solution" or "Current State → Fix"?: For new features, purpose/motivation is also acceptable. However, a one-sided description only listing "what was done" is insufficient.
- Is the scope of changes clear?: Can you tell what was changed and what was not changed?
- Is the implementer's own understanding visible?: Does it convey what the implementer was thinking when making this change, rather than just pasting AI-generated text as-is?
Common Policy for Consulting External Agents
In Steps 3, 4, and 5, seek a second opinion from external Agents (however, the subsection "Consider Naming and Design from a Long-Term Perspective" in Step 4 is excluded). In Step 6, consult external Agents only if there are questionable points. Follow the fallback order below, and actually call and test instead of judging by speculation:
If you yourself are Codex CLI, start from 2. Otherwise, start from 1.
- Call using the Skill tool. If it fails, proceed to 2.
- Call using the Skill tool. If it fails, proceed to 3.
- The main agent performs the work alone.
If a fallback occurs or external Agents are unavailable, document this in the "Issues Encountered During Review Work" section of the report.
If fallback to 3 (main agent solo execution) occurs, it means the chain of critical thinking that is the core of this skill (a protocol where parties examine and refute each other's claims to improve accuracy and comprehensiveness) is not functioning. A review without reciprocal verification does not have its intended accuracy, so be sure to include the following text as a bold paragraph at the beginning of the "Issues Encountered During Review Work" section of the report:
⚠ Warning: The chain of critical thinking is not functioning. Question whether the execution environment is sound.
Each step describes the content of Args passed to external Agents and how to handle the results.
Step 3: Verify Consistency Between Implementer's Description and Implementation
The purpose of this step is cross-referencing documentation and code to confirm that the descriptions in the PR description and comments match the actual code.
Important: Only Collect Statements from the Implementer
Cross-reference the author of the PR description with the author of each comment, and treat only the implementer's own statements as descriptions of the implementation.
Supportive comments written by others, comments expressing expectations for features, questions, etc., are not descriptions of the implementation.
Confusing these with implementation descriptions will lead to incorrect consistency judgments.
Check Items
- Does the description in the PR match the actual diff?
- Do the implementer's explanations in inline review comments match the actual code?
- Do the implementer's explanations in the PR review body (top-level review comment) match the implementation?
- Does the content of the conversation context match the implementation? (if conversation context exists)
If discrepancies are found, record them specifically.
Consistency Check by External Agents
In addition to the Agent's own checks, have external Agents also verify the consistency between the diff and the description.
Having another perspective on the code may uncover discrepancies that the Agent missed.
Follow the "Common Policy for Consulting External Agents" and call with the following Args:
Args: Please consult thoroughly. I want you to check if there are any discrepancies between the description in PR #{number} and the actual diff. {summary of description and check points}
After receiving the external Agent's feedback, cross-reference it with your own check results to confirm if there were any oversights.
Step 4: Consistency of Naming and Design Patterns
The purpose of this step is understanding the codebase by confirming that the implementation matches the conventions of the existing codebase.
Understanding the codebase before investigating bugs and vulnerabilities improves the accuracy of subsequent investigations.
Check Items
-
Consistency of Naming Conventions
- Do file names, function names, variable names, and class names match the naming patterns of existing code?
- Have names that omit parts of the official feature name been introduced?
-
Consistency of Design Patterns
- Do the structure, module division, and responsibility allocation match those of existing similar features?
- Are existing abstractions utilized, or is there duplicate implementation?
If discrepancies are found, record them specifically.
Verification by External Agents
In addition to the Agent's own checks, have external Agents also verify the consistency of naming and design patterns.
Follow the "Common Policy for Consulting External Agents" and call with the following Args:
Args: Please consult thoroughly. I want you to check if the naming and design patterns added/modified in PR #{number} match the conventions of the existing codebase. {change summary}
After receiving the external Agent's feedback, cross-reference it with your own check results to confirm if there were any oversights.
Consider Naming and Design from a Long-Term Perspective
Consider whether the current naming and design could become a source of problems when the codebase is expanded in the future.
Unlike consistency with existing code (a factual judgment), this is an exercise to provide seeds for discussion and stimulate human imagination.
If there are points of concern, write them out in a self-questioning format that covers "concerns, counterarguments, conclusion or reservation".
Since this is the Agent's own opinion, it should be written as regular paragraphs. About 2-3 paragraphs per point is sufficient.
You don't have to force a conclusion. If there are no points of concern, write "None in particular".
This consideration section is not assigned to external Agents; the Agent executing the skill thinks about it themselves.
Output example:
markdown
The name "access-token" is not a problem currently because it only refers to tokens between users and services.
However, if tokens for connecting with external services are introduced in the future, there is a possibility of conceptual conflict.
When that happens, it may become necessary to include the subject in the name, like `user-access-token`.
However, since there are currently no visible plans to add external service integrations in the near future, I judge that this is not an issue that needs to be fixed immediately.
The name "member-metrics" also has room for ambiguity if the concept of "member" expands beyond project members in the future.
On the other hand, in existing code, `Member` is established as a term referring to project members, so it seems unlikely to be a major problem in the short term.
Therefore, I judge that there is no problem at this time.
However, if other types of member concepts such as organization members are introduced in the future, a review of the naming will be necessary.
Step 5: Investigation of Bugs and Vulnerabilities
Follow the "Common Policy for Consulting External Agents" and request a code review from external Agents.
Include the PR change summary, diff summary, and points to check in the arguments.
Be sure to include a depth specification in the arguments. Consultation skills will ask the user back if the depth is not specified, which interrupts the review flow:
Args: Please consult thoroughly. Please conduct a code review of PR #{number}. {change summary and check points}
Result Verification
Do not blindly accept the external Agent's feedback. Do the following:
- Verify the points raised by the external Agent by reading the code yourself
- Consciously look for areas that the external Agent may have overlooked
- If the external Agent's feedback differs from your own opinion, record the reasons for both
Step 6: Re-read Conversation Context and Check for Oversights
If there is no conversation context, skip this step.
Code review is not a double-check of work results. Review the process, not just the result.
What was done can be understood by reading the code. What was not done after consideration is the information necessary for design review.
Verify whether the "process" of design decisions, rejection reasons, and intentional non-implementation written in the conversation context is correct.
Re-read the conversation context carefully and verify the following:
6-1. Basis for Design Decisions
Do the reasons for design decisions written in the conversation context match the actual state of the code?
6-2. Re-evaluation of Rejected Alternatives
Extract each alternative one by one from the "Rejected Alternatives" section of the conversation context and re-evaluate them:
- Is the reason for rejection valid? Was the comparison truly fair?
- Are there any overlooked advantages or overestimated disadvantages?
- Are the prerequisites used to compare the adopted and rejected proposals correct?
6-3. Process Verification of Failed Attempts
For items described as "tried but didn't work", question not just the result but the way it was tried itself:
- Were the prerequisites correct?
- Were there any oversights in the execution procedure?
- Was the judgment criterion (the basis for determining "it didn't work") valid?
- Is there a possibility that the result would change if tried under different conditions?
6-4. Validity of Intentionally Unaddressed Items
For items decided to "not do":
- Is the reason for "not doing" still valid considering the current implementation?
- Has the premise changed as a result of the implementation, making it something that "should be done"?
6-5. Accuracy of Stated Facts
Verify whether the "facts" written in the conversation context are truly correct by reading the code.
If there are questionable points, consult external Agents according to the "Common Policy for Consulting External Agents". Be sure to include "Please consult thoroughly" in the Args.
Step 7: Create the Review Report
Load TEMPLATE.md located in the same directory as this SKILL.md, and create the report according to its format.
The report will be output to the conversation.
If it was decided in Step 0 to post a comment to the PR, post the same content as a comment on the PR.
Report Creation Guidelines
- pull request description > Summary: Organize the implementer's explanations in a before-after format by quoting and excerpting. Do not make up content based on imagination. If the implementer's explanation is insufficient, point that out honestly.
- pull request description > Quality Assessment: Fill in the checklist results from Step 2 as OK/NG/Not applicable. If NG, specify what is insufficient.
- Issues Encountered During Review Work section: If review steps were skipped, document whether it was due to external factors (e.g., tools unavailable) or an Agent decision. If there are no issues, write "None in particular".
- Conclusion section: Describe the overall comprehensive judgment and recommended actions.
- Throughout the report, keep in mind that the goal is to provide materials for reviewers to judge "whether this change is valid".
Related Skills
- codex-consultation: Skill for consulting with Codex CLI. Used as the first choice for external Agents in Steps 3, 4, 5, and 6.
- subagent-consultation: Skill for consulting with Agent tools (subagents). Fallback destination when codex-consultation is unavailable.
- conversation-context-import: Skill for loading conversation context. Background knowledge for Step 1.
- conversation-context-export: Skill for exporting conversation context. Background knowledge for conversation context format.
- library-library: library for library library library?