Babysit a PR
Goal: carry a pull request (GitHub) or merge request (GitLab) from "just opened" to "nothing left
unanswered," without the human having to sit and refresh the page. "PR" below means either.
Review bots are diff-anchored samplers. Every push mints a fresh round, and a fix in one place can
light up commentary somewhere adjacent. Left alone, a PR accumulates half-answered threads that
nobody resolves, and the real bug in round three gets buried under nitpicks from rounds one and two.
Your job is to be the person who reads every finding, decides what is actually true, fixes what
blocks, and closes every loop in writing.
You know how to drive
(GitHub),
(GitLab), and git. What follows is only the judgment this
loop needs and the few API calls that are easy to get wrong. The harvest script picks the forge from
the cwd's git origin; everything it returns has the same shape on both, with a
block
naming what that forge cannot tell you.
The three rules that matter most
Verify before you believe. A bot's severity badge is a guess made without running anything. Treat
every finding, including the P1s, as a claim to check against the code. Bots are frequently right
(that is why this loop is worth running), and they are also confidently wrong often enough that
shipping their suggestions unexamined will introduce bugs. Read the actual code path before you
agree or disagree.
Every thread gets an answer. A finding you fixed, rejected, or deferred is only closed once you have
said so in that thread and resolved it. Silence reads as "ignored" to the next human who opens the
PR, and it is how a real bug gets lost.
Publish before you answer. A "fixed" reply is only true once the remote branch carries the fix.
Never post a confirmed reply, or resolve its thread, while the fix exists only locally. Rejections
need no push. Reply with evidence and resolve immediately.
Harvest the round
Findings arrive on two different surfaces, and a round that reads only one silently misses half of
them. This is the single most common way a babysit loop goes wrong:
- Inline review threads. This is where debate-review and Codex post their findings (Codex attaches
P1/P2-badged inline comments to an otherwise boilerplate review body; an empty-looking body proves
nothing). Each thread carries a (to resolve) and a (to reply inside the
thread). On GitHub these are GraphQL review threads; on GitLab they are discussions.
- Top-level review bodies. This is where Greptile summarizes, Codex sometimes posts a numbered list,
and debate-review posts its round summary. These have no thread to resolve; answer them with one PR
comment per round. On GitHub they are review objects; on GitLab they are plain notes.
The bundled script returns both in one call, already correlated (
is the folder that
holds this SKILL.md):
bash
"<skill-dir>/scripts/threads.sh" <N> > /tmp/pr-<N>-round-<k>.json
Never trust a filtered count without its unfiltered twin. Before applying any jq filter to the
harvest, print the raw totals (
jq '{threads: (.threads|length), reviews: (.reviews|length)}'
) and
compare. A filter that eliminates 100% of items is presumed broken until the field names are
verified against the actual schema (
). jq selects on a misspelled field
fail silently-empty, and a "clean round" built on one is how a P1 gets a merge-gate mention posted
over it. That has happened. GitHub tooling fails by returning less data, not by erroring; pair this with the
pagination rule.
Never describe an object you did not fetch. If a query for a specific id returns empty, that is a
stop signal. Say "I can't see it" and fetch it another way (
), never narrate
its presumed content. Related trap: every inline thread reply arrives wrapped in a zero-byte
review object, so a watcher's "new review" event may be just a reply wrapper, not a new
round. threads.sh's
does not include these wrappers, so a review id from an event that is
missing from the harvest means "wrapper", not "gone".
Diff it against the previous round's file to see what is genuinely new.
on a thread
means the line moved underneath it. The finding may already be fixed, so check it against current
code before spending the round on it. A
bump on a thread you already handled means a
bot followed up inside it.
Has this reviewer seen the current push? Only trust a field that names a sha. On GitHub each review
carries
; compare it to
. For debate-review on either forge, the round body's
is the sha it reviewed. On GitLab other reviewers' notes carry no sha (
capabilities. review_commit_id: false
); a note's timestamp being later than your push does not prove it reviewed
that push, so say "coverage unknown" rather than guessing.
Two kinds of author count as a reviewer. First, a bot:
in the harvest. On GitHub
that comes from the API's own author type and is reliable (
and
are the usual ones; don't hardcode a whitelist). On GitLab the API only sometimes
says, so
can be
; treat
as unknown, look at the thread, and say in your
report that you could not confirm it. Second, any thread whose first comment carries a
<!-- debate-review:... -->
marker. debate-review posts from the user's own account, so the author
is the PR author (
author_is_pr_author: true
), but the thread is a reviewer thread. The harvest
flags these as
with
,
, and
parsed
from the marker; its round body shows up in
with
(the sha it reviewed) and
/
. Treat them like any other bot thread. Anything else from the PR
author, and any human's comment without that marker, is never in scope for autonomous fixing.
Surface it to the user instead.
Bots post 5 to 10 minutes after a push, longer on a big diff. Don't poll tightly; background the wait
and review the diff yourself meanwhile. A round is "in" once every reviewer you expect has either
posted against the current head SHA or been marked unavailable after its own wait budget. An
unavailable reviewer never blocks harvesting or acting on the ones that did post. Disclose the gap
instead of reporting the PR clean.
Classify by real impact, not by badge
After verifying a finding, sort it by consequence rather than by the label the bot attached.
Blocking, meaning it would ship a defect or stop the merge:
- a real bug, wrong behavior, or broken edge case in the changed code
- security, authorization, data-integrity, or data-loss exposure
- a violation of the change's own stated contract, acceptance criteria, or spec
- a migration or schema hazard
- a failing or newly-flaky check
Non-blocking, meaning real but ships nothing broken: naming, structure, docs, test nitpicks, micro
performance, "consider extracting this", style preference.
When a finding is genuinely ambiguous, hold it as blocking until you have read enough code to demote
it. The asymmetry is deliberate. An over-cautious fix costs minutes, a missed P1 costs a production
bug.
A debate-review thread with
means two models looked and disagreed. The
main reviewer held the finding against a refutation, and the italic last line of the comment says
what the challenge was. That is a claim with a known counter-argument, not a weaker claim. Verify it
the same way, and say in your reply which side the code supports and why.
Fix the blockers, autonomously
Don't stop to ask about blockers. Verify, fix, push, keep watching, report what you did.
- Reproduce first where you can. A probe that fails before the fix and passes after is what separates
a real fix from a plausible edit. This matters most on findings you initially disagreed with. Those
are the ones where being wrong is expensive.
- One push per round, not one per finding. Every push mints a new bot round, so per-finding pushes
multiply the rounds you have to sit through.
- Run the repo's own gate before pushing. A fix that breaks the suite costs a whole extra round.
- If a matching guard skill is installed (clean-code-guard, test-guard, wp-guard, woo-guard from
guard-skills), run it on your fix before pushing. The guards catch the failure modes a quick fix
under review pressure tends to produce.
- When you disagree, prove it. Rejecting a finding is legitimate and common, but the reply has to
carry the evidence: the code path, the guard that already handles it, or the test that pins the
behavior. "This is fine" is not a rejection.
Publish, then reply, then resolve
Work the round in one pass, not per finding: verify everything, reproduce confirmed blockers where
practical, fix them all, run the gate, then commit and push once and confirm the remote SHA. Only
then close the loops:
- Confirmed: reply naming the fix commit, then resolve.
- Rejected: reply with concrete evidence, then resolve. No push needed; these can close anytime.
- Deferred: create the agreed issue, reply with its link, then resolve.
Non-blocker fixes the user approves ride the next consolidated push, never a dedicated push of their
own. There is no re-review-exempt push: every push, including a final docs-only or nit-only one, must
be covered by a clean round from the merge-gate reviewer before merge (see "Before merge"). If
publication or verification fails, leave the thread open and report the blocker.
Answer inside the thread the finding came from. A fresh top-level comment leaves the original thread
open and forces the reader to correlate by hand. Use the harvest's
to reply and
to resolve. On GitHub those are two different identifiers (REST comment id, GraphQL thread id); on
GitLab both are the discussion id.
GitHub:
bash
gh api --method POST "repos/<owner>/<repo>/pulls/<N>/comments/<reply_to>/replies" \
-f body="$(cat /tmp/reply.md)"
gh api graphql -f query='mutation($t:ID!){
resolveReviewThread(input:{threadId:$t}){ thread{ isResolved } } }' -F t="<thread_id>"
GitLab (
is the URL-encoded
,
your instance):
bash
glab api --hostname <host> --method POST "projects/<project>/merge_requests/<N>/discussions/<reply_to>/notes" \
--raw-field "body=$(cat /tmp/reply.md)"
glab api --hostname <host> --method PUT "projects/<project>/merge_requests/<N>/discussions/<thread_id>" \
-F resolved=true
The GitLab reply and resolve calls are taken from the GitLab API docs and have not yet been exercised
against a live instance from this skill. The first time you use them, check the response, and if
either fails, stop and report rather than retrying variations.
Attribution. Open every reply by naming the model writing it and the person it writes for, so a
reader never has to guess whether a human weighed in. Sign your own model name; this skill is
model-neutral. The person is whoever owns the account the reply posts from. Get the name once per
session,
gh api user -q '.name // .login'
on GitHub or
glab api user --hostname <host> | jq -r '.name // .username'
on GitLab, and reuse it:
I am <model-slug> writing on behalf of <user>.
Then the verdict, then the evidence, briefly:
I am <model-slug> writing on behalf of <user>.
Confirmed and fixed in `a1b2c3d`. You were right that `occurrence_time` was never
compared against `evidence.event_time`, so a mapping could bind proof from a
different occurrence. Reproduced with a failing test first
(`test_binds_proof_to_mapped_occurrence`), then fixed the composition check.
I am <model-slug> writing on behalf of <user>.
Declining this one. The nil case you describe is already unreachable. `resolve()`
returns early at `handlers.py:88` whenever the session is unset, which is the only
path that reaches this line. Leaving the behavior as-is.
Resolve only what is actually closed: fixed and pushed, rejected with evidence, or deferred with an
issue filed. Never resolve a thread whose question you have not answered. Resolution claims the loop
is closed, and a false claim is worse than an open thread.
Non-blockers: one batched ask per round
Don't interrupt per finding, and don't silently decide. Once per round, after the blockers are
handled, bring the non-blocking findings as one list with a recommendation each (fix now, open an
issue, or reject) and let the user choose:
Round 2 on PR #123. 1 blocker fixed and pushed (
). Three non-blocking findings left:
- debate-review: extract the duplicated fixture in . Recommend issue, touches
files outside this change
- Codex: comparison could use directly. Recommend fix now, one line
- Greptile: docstring missing on the new helper. Recommend fix now, trivial
Fix 2 and 3 in the next push, issue for 1?
Whatever they decide, close each thread the same way as any other finding. Anything deferred gets a
real issue with enough context to act on months later: a link back to the thread, the file, and why
it was deferred. Not just a title.
Re-trigger within a fixed budget
One invocation gets the initial harvest plus at most two consolidated repair pushes and two
re-review cycles unless the user explicitly asks to continue. After each push you start the next
round yourself. How depends on the reviewer, because they are triggered in three different ways:
-
debate-review is a local script, not a bot, and it works on both forges. You run it, it does the
whole review while you wait, and it exits once the review is posted. Nothing to mention, nothing
to poll:
bash
node "<debate-review skill-dir>/scripts/review-pr.mjs" <pr-url>
(
<debate-review skill-dir>
is wherever that skill is installed,
~/.agents/skills/debate-review
on a standard install.) Run it in the background, keep working, and harvest the moment the command
exits. It prints the review URL; exit code 3 means this head was already reviewed. It reviews
exactly one head sha per run, so a run after a push always produces a fresh round. A run takes
10 to 20 minutes.
-
Codex is a GitHub app (there is no GitLab equivalent). Mention
in a PR comment,
then wait. It answers 8 to 15 minutes later, against whatever head was current when it ran. Check
on its review before believing it covers your push.
-
Greptile and similar bots re-review every push on their own. Don't summon them; handle their
findings when they show up.
For a bot you are waiting on, wait at most 10 minutes past its usual window. If it is silent or
rate-limited, mark that reviewer unavailable; do not wait out a cooldown. The one exception is the
merge-gate reviewer at the merge gate, which has no timeout (see "Before merge"). Even a final
test-only, documentation-only, or nit-only push gets a round. The merge gate below is meaningless if
the last push went unreviewed.
Run the repository's required gate once per consolidated repair push; never rerun an already-passing
gate for the same SHA. If the user says "stop", "enough", or "push whatever you have", cancel active
polls and long gates, run the smallest relevant check that can finish promptly, publish the safe
work, disclose any incomplete gate, and do not trigger another review round.
Rounds should shrink. If round three is as large as round one, something systematic is wrong. Say
so rather than grinding. Findings that recur in the same shape usually mean the fix addressed a
symptom instead of the cause, which is worth surfacing.
At the budget boundary, stop and hand off the exact remaining findings, unresolved threads, last
reviewed SHA, and unavailable reviewers. Never describe an unreviewed head as clean.
Before merge
The merge gate is an explicit clean round from the repo's primary reviewer on the exact merge
candidate, the final head sha. Which reviewer that is depends on the repo.
Where Codex is installed, mention
after the final push and wait for its reply. A
clean round is Codex saying so in plain words ("no findings", "good job") against the final head. No
reply yet is not a pass. Codex answers 8 to 15 minutes after a push, and merging inside that window
is how a real finding lands minutes after the merge.
Where debate-review is the reviewer (on GitLab it is usually the only one), run it on the final head.
A clean round is all three of: its round body present in
with
equal to the
final head sha,
and
both zero, and no unresolved reviewer threads.
If the body is missing (a run can fail after posting inline comments), the gate has not been met;
re-run it, don't infer.
Either way, a finding is a new round, not a merge. Silence well past the usual window is something
to report to the user, not approval.
Re-harvest and re-read the PR's most recent comments before proposing a merge. A watcher settled
into a quiet interval can miss a late round, and a comment posted after your last check is exactly
the one that gets merged over.
Then ask the user whether to merge. Never merge on your own initiative. Report: rounds run,
blockers fixed with their SHAs, findings rejected and why, issues filed, unresolved threads
remaining (ideally zero), and CI state. The merge decision is theirs; everything leading to it was
yours.
When to stop and speak up
Some situations are not yours to grind through:
- A bot finding that is right but demands a change well beyond this PR's scope.
- Two bots contradicting each other on the same line, when code, tests, and the stated contract
cannot settle it.
- The same finding recurring after a retry. The first recurrence gets a re-verified root cause and
one more attempt inside the repair budget; a second means your model of the bug is wrong.
- A human reviewer's comment, always.
- CI failing for infrastructure reasons rather than code.
- The two-repair-cycle budget is exhausted.
- The user asks to stop, push the current work, or end the babysit loop.