Validate a security patch against the reported bug and the surrounding code it affects. Give the patch author reproducible failures to fix and identify what still needs testing. Apply the same checks to human and agent patches. The diff, author, upstream implementation, and original proof of concept alone cannot establish correctness.

The runner rejects incomplete plans. Supply at least one check of every kind:

control

exploit

variant

behavior

regression

security

suite

Commands are argv arrays, never shell strings. Put complex setup in a checked-in or plan artifact script and invoke it with . The runner fixes locale/timezone/hash-seed inputs, executes checks in lexical ID order, records raw stdout/stderr, and never edits the original worktree. Each check's defaults to 300 and accepts integers from 1 through 3600. Exceeding the timeout leaves a validation gap. A timeout alone does not establish a regression. Every plan also contains a sorted array ( when none). Scaffolding infers affected Gitlinks from the changed-file inventory. The runner initializes those pinned commits from the source repository's existing Git module objects, never from network URLs; initialize or fetch them in the source repository before validation.

A nonzero exit does not mean the vulnerability reproduced. An import error, a failed build, a missing dependency, and a failed safety assertion all exit nonzero and are indistinguishable to the runner. Every and check must print and flush immediately before it evaluates its assertion, on both revisions:

"argv": ["python3", "-c", "import app; value = app.render('<'); print('PPV_REACHED', flush=True); assert value == '&lt;'"]

The token is also in the environment as . It must land on stdout, as a line of its own. Stderr is not scanned, because a Python traceback echoes the offending source and would otherwise satisfy the check for a harness that executed nothing. A run without it leaves a gap. Independent findings from other checks remain in the result. Flush explicitly: a harness whose payload segfaults or calls loses buffered output and forfeits its own evidence.

These checks also run side-blind. is not expanded for them, is absent from their environment, the checkout directory is randomly named, and the plan validator rejects any or check whose or mentions either. An assertion that can see which revision it is on can assert on that instead of on the code, which is the cheapest possible way to fake a reproduction followed by a fix.

Checks run under a fixed minimal environment: , , and a handful of temp/user keys, plus /, , , , . Everything else in the caller's environment is dropped. Toolchains that need more get it explicitly:

uv run {baseDir}/scripts/post_patch_validation.py run \
  --plan post-patch-validation/plan.json \
  --output post-patch-validation/results \
  --allow-env JAVA_HOME --allow-env CARGO_HOME

Forwarded names and values are recorded in . A requested variable that is unset is an error, not an empty string. Two classes are refused outright: names that read as credentials (, , , …), because the value would be written into the result; and names that change what executes (, , , , …), because those variables can change which code executes. The runner's fixed variables and every name are also reserved and cannot be forwarded.

Placeholders expanded in and per-check values: (the revision under test), (an isolated copy of the plan artifacts for that one invocation), (a fresh opaque directory for that one check invocation), and ( or , and not available to / checks). The same values arrive as , , , , and . Write only under ; the evidence directory path is not passed to checks. Base and patched invocations do not share runner-managed scratch, plan, or worktree roots. After each invocation exits, its scratch tree is archived under the deterministic path, its private plan copy is discarded, and every readable argv element that resolves to a file is hashed in . Files inside the isolated plan or checkout roots are additionally retained under up to 16 MiB; the record explains why any other file was not archived. Use a dedicated directory for : its sibling files and directories are copied into each invocation's . Keep helper code under that directory's directory or checked into the target repository so its bytes are reviewable. The machine plan containing commit pins, the current output directory, and detected prior result trees are excluded; symlinks are rejected. The clean snapshot remains only in runner memory, and exploit/variant sides execute in random order while evidence filenames remain deterministic. Stdout/stderr use anonymous or randomly named capture descriptors and are copied to the named evidence files only after the child exits, so fd inspection cannot disclose the side label.

This isolation is not a host sandbox: checks run with the caller's privileges and a malicious helper could use arbitrary external state or deliberately infer the revision from source or Git metadata. Inspect the content-addressed helper artifacts, and use an OS/container sandbox when the check code itself is untrusted.

Active validation worktrees are Git-locked with random owner tokens backed by kernel file locks, so another concurrent validator cannot prune them and PID reuse cannot impersonate an owner. If the runner is forcibly killed, the next run unlocks stale validator-owned registrations. For manual recovery, inspect , then use and (or after the path is gone).

Read evidence-model.md when designing coverage, selecting variants, or interpreting findings and validation gaps. Do not read it for routine CLI execution.

schema 2.0 contains an with , , , and . Status is when all required checks produced usable evidence, even if some checks found failures. Status is when any gap remains. Findings name the check ID, check kind, and failed expectation. Gaps name the check ID and missing evidence, with a null check ID for run-level problems such as cleanup failures.

Suite checks run on the patched revision first. A completed failure triggers the same check on baseline. If baseline passes, report the failure after the patch. If both fail, preserve both logs and report that attribution is unresolved. Do not assume matching exit codes mean the same failure.

The runner exits 0 for complete checks with no findings, 1 for complete checks with findings, 10 for incomplete validation, and 64 for invalid inputs. Read the artifact even after exit 10: it can contain supported findings alongside gaps. Source or build evidence cannot establish runtime behavior. State the evidence level alongside any passing result.

Claude Code exposes the bundled workflow as . To pass structured inputs through the tool, use the name without a leading slash:

Workflow({
  name: 'post-patch-validation:validate-patch',
  args: {
    finding: '<finding text or local path>',
    baseRef: '<vulnerable-ref>',
    patchRef: '<patched-ref>',
    workdir: 'post-patch-validation',
  },
})

Use instead of when appropriate. The workflow uses fixed coverage lenses to propose checks and a fixed executor to run this skill. Agents may author test artifacts. The Python runner records findings and gaps, and reviewers report coverage or evidence concerns separately. returns supported failures even when other checks left gaps. means evidence is incomplete without a supported failure. Passing checks advance to only after both evidence reviews approve, otherwise . The workflow cannot ask questions after launch, so pass every input up front.