evaluate-ai-change
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChineseEvaluate an AI change
评估 AI 变更
The skill gives the agent evaluation discipline. EvalGate supplies execution,
evidence, history, permissions, and enforcement.
该 skill 为 Agent 提供评估规范。EvalGate 提供执行、证据、历史记录、权限和强制管控。
CLI invocation
CLI 调用
Every command in this skill means . Never run
— the unscoped npm package is a third-party package. Discover the installed Python CLI before choosing it. If the requested command
is absent, run even in Python repositories. Do not equate a
source-tree Python version with a published package.
evalgatenpx @evalgate/sdknpx evalgatenpx @evalgate/sdk本 skill 中的每个 命令均指 。切勿运行 ——未限定 scope 的 npm 包是第三方包。在选择已安装的 Python CLI 前,应先发现它。如果请求的命令不存在,即使在 Python 仓库中也要运行 。不要将源码树中的 Python 版本等同于已发布的包。
evalgatenpx @evalgate/sdknpx evalgatenpx @evalgate/sdkDecide whether evaluation is required
判断是否需要评估
Inspect the requested diff and the repository before running anything. Invoke an
EvalGate workflow when the change can alter observable behavior, including:
- prompts, system instructions, model/provider choice, parameters, or routing;
- agent orchestration, tools, schemas, permissions, skills, or MCP integrations;
- retrieval, context, memory, structured output, judges, datasets, or harnesses;
- business logic on an AI execution path; or
- quality, safety, reliability, latency, cost, or release-gating behavior.
Do not run EvalGate merely because a repository contains AI code. If the change
is demonstrably isolated from observable AI behavior, record that reasoning and
continue with the repository's ordinary tests. When uncertainty is material,
start with impacted coverage and expand only as the evidence requires.
Read
when impact or scope is ambiguous.
references/behavioral-change-detection.md在运行任何命令之前,检查请求的 diff 和仓库。当变更可能改变可观察行为时,调用 EvalGate 工作流,包括:
- prompts、system instructions、model/provider 选择、parameters 或 routing;
- Agent 编排、tools、schemas、permissions、skills 或 MCP integrations;
- retrieval、context、memory、structured output、judges、datasets 或 harnesses;
- AI 执行路径上的 business logic;或
- quality、safety、reliability、latency、cost 或 release-gating 行为。
不要仅因为仓库包含 AI 代码就运行 EvalGate。如果能够证明变更与可观察 AI 行为相隔离,请记录该判断依据,并继续执行仓库的常规测试。当不确定性具有实质影响时,先从受影响范围的 coverage 开始,仅在证据需要时扩大范围。
当影响或范围不明确时,阅读 。
references/behavioral-change-detection.mdInspect before acting
执行前检查
- Read repository instructions and preserve its package manager and test flow.
- Run . Treat durable repository linkage, checkout relation, cloud snapshot identity, local-gate readiness, and cloud-target readiness as separate facts. Branch movement does not unlink a repository; never reuse cloud evidence unless the exact target SHA and manifest match.
npx @evalgate/sdk status --json - When a local checkout is present, locate , evaluation files, datasets, reviewed baselines, quality profiles, traces, judge configuration, experiment history, and CI.
evalgate.config.json - When a local checkout is present, run
to preview an evidence-backed product theory. Treat detections as hypotheses, not runtime facts.
npx @evalgate/sdk understand --format json - State the intended behavior change, affected surfaces and slices, existing coverage, missing evidence, and the minimum sufficient evaluation scope.
- Ask for approval before writes, tests with meaningful cost or side effects, baseline changes, cloud access, credential creation, or release mutations.
For a response, document, table, image, structured prediction, retrieval result,
or other non-repository artifact, do not require a checkout. Inspect the host's
actual capabilities first: readable inputs, local tools, shell, configured API
or MCP tools, authenticated principal, upload support, completion/event hooks,
asynchronous delivery, and stable artifact/revision identity. Capability and
permission are separate facts. Read
for the bounded workflow and its
unsupported states.
references/portable-artifact-workflows.mdSteps 3 and 4 are conditional on purpose. A durably linked repository can be
inspected without a checkout, so requiring a local config before routing would
force a clone on a caller whose question the repository commands already
answer. If there is no checkout, skip to the routing table below and let intent
and authorized context decide.
Use when new coverage is
needed. Do not invent a parallel configuration when the repository already has
a canonical EvalGate workflow.
references/eval-authoring.md- 阅读仓库说明,并保留其 package manager 和测试流程。
- 运行 。将持久仓库关联、checkout 关系、云快照身份、本地 gate 就绪状态和云目标就绪状态视为彼此独立的事实。分支移动不会解除仓库的持久关联;除非目标 SHA 和 manifest 完全匹配,否则绝不能复用云端证据。
npx @evalgate/sdk status --json - 存在本地 checkout 时,找到 、evaluation 文件、datasets、经审阅的 baselines、quality profiles、traces、judge 配置、实验历史和 CI。
evalgate.config.json - 存在本地 checkout 时,运行 ,预览有证据支持的产品理论。将 detections 视为假设,而非运行时事实。
npx @evalgate/sdk understand --format json - 说明预期行为变更、受影响界面和切片、现有 coverage、缺失证据,以及最小充分评估范围。
- 在执行写操作、具有显著成本或副作用的测试、baseline 变更、云端访问、凭据创建或发布变更之前,请求批准。
对于响应、文档、表格、图片、结构化预测、retrieval 结果或其他非仓库产物,不要求必须存在 checkout。首先检查宿主的实际能力:可读输入、本地 tools、shell、已配置的 API 或 MCP tools、已认证主体、上传支持、completion/event hooks、异步交付,以及稳定的 artifact/revision 标识。能力和权限是彼此独立的事实。阅读 ,了解有界工作流及其不支持的状态。
references/portable-artifact-workflows.md步骤 3 和 4 视目的而定。持久关联的仓库无需 checkout 即可检查,因此在路由前强制要求本地配置,会迫使那些问题本可由仓库命令回答的调用方进行 clone。如果没有 checkout,请跳至下方路由表,由意图和已授权上下文决定后续流程。
需要新增 coverage 时,使用 。当仓库已有规范的 EvalGate 工作流时,不要另建并行配置。
references/eval-authoring.mdStandalone installation
独立安装
This skill bundles its decision schema and references. Load a routed sibling from
when installed; otherwise use its canonical source link
below. Do not assume sibling files exist. The primary loop remains usable offline:
inspect capabilities and the existing evaluation flow, run the authorized gate,
and report its evidence without requiring an additional install.
../<skill-name>/SKILL.md此 skill 打包了决策 schema 和参考资料。安装后,从 加载路由选中的同级 skill;否则使用下方规范源链接。不要假设同级文件一定存在。即使离线,主流程仍然可用:检查能力和现有评估流程,运行已授权的 gate,并报告其证据,而无需额外安装。
../<skill-name>/SKILL.mdSelect the workflow
选择工作流
Route on what the caller asked for and what their context already authorizes.
The absence of a scaffold is not a reason to build one: it usually means nobody
has looked at the repository yet, and looking is read-only.
Establish two facts first, because they decide the branch:
- Intent. Understand something, evaluate a specific change, gate a release, or get reference material? Take the caller's stated intent over an inferred one.
- Authorized context. Is there a durable repository linkage and a scoped
credential (plus
npx @evalgate/sdk whoami), a local checkout, both, or neither? Never assume authority that has not been demonstrated. Linkage, checkout relation, and an exact cloud-targetable snapshot are three separate facts: branch movement, uncommitted files, and unpushed commits do not erase a durable linkage, though they can mean the current contents are not a verified cloud target. A savednpx @evalgate/sdk status --jsonsession orloginboth count;EVALGATE_API_KEYshows which is active.whoami
Then:
| Intent | Authorized context | Workflow |
|---|---|---|
| Understand a system, or scope a change before touching it | Durable linkage + EvalGate credential, evidence for the exact target snapshot already exists | ask-repository-question — no scaffold, no GitHub repository-write access |
| Understand a system | Durable linkage + read scope, but no evidence for the required snapshot | Name the missing evidence and the exact authority a scan needs. Do not answer from a different snapshot and do not guess. |
| Understand a system | No linkage yet | Offer the supported identity, consent, and linkage handoff. No local scaffold is required to get there. |
| Understand a system | Local checkout only | |
| Create the missing evidence | EvalGate | Start a scan. This mints EvalGate evidence; it does not need and must not request GitHub repository-write access. |
| Evaluate a specific change | Reviewed scaffold and baseline exist | run-regression-gate |
| Gate a release, and the caller named an existing gate | That gate exists | Use it directly. Do not route a caller with a working gate through onboarding. |
| Evaluate a specific change | No reviewed scaffold, and the caller wants durable coverage | setup-evalgate-project |
| Inspect an AI-generated change or external optimization result | Local artifact available | Discover whether |
| Inspect or run an improvement cycle | Scoped hosted context | Discover |
| Confirm a candidate | Frozen candidate and eligible confirmation evidence | Inspect |
| Explain behavior a static read cannot | Runtime evidence missing | collect-agent-traces |
| Get reference material | Public documentation only | use-evalgate-mcp — no signup |
| Get scoped product state | A read scope | use-evalgate-mcp |
| Check a response or non-repository artifact | Host can read the exact artifact locally | Use this Skill's portable artifact workflow; do not require repository setup |
| Check a response or non-repository artifact | No shell, but an authorized API/tool advertises the required operation | Use the advertised operation and preserve its task, artifact, revision, and result identifiers |
| Check a response or non-repository artifact | Neither a local evaluator nor an authorized operation is available | Report the exact setup/capability gap and unresolved evidence; do not claim execution |
| Evaluate coding work with no AI on the product path | Inspection proves observable AI behavior cannot change | Classify |
| Start or receive background checks | The host exposes a verified event hook and delivery path, and a bounded grant authorizes both | Use only the advertised hook and authorized scope; otherwise keep evaluation explicitly invoked or post-response advisory |
| Reuse an approved standard in another host | Same tenant and attributable delegated person, authorized scope, exact standard version, and retrieval operation are verified | Retrieve that version and preserve an independent second-task result; otherwise report the missing identity, permission, or capability |
Prefer the least authority that answers the question. A durably linked
repository with no scaffold and no GitHub repository-write access can already
reach evidence-linked understanding through ; do not
route such a caller into scaffolding or baseline acceptance to answer a
question the repository commands already cover. The EvalGate scopes differ by
step: reading existing evidence needs , while starting a scan needs
and membership because it mints new evidence. Neither is GitHub
repository-write access.
npx @evalgate/sdk repoeval:readeval:writeEscalate only when the intent requires it, and only with the specific authority
that step needs. Reading does not authorize executing, executing does not
authorize a patch, a patch does not authorize a push, and a passing gate does
not authorize a merge or a pull request. Opening or merging a pull request is
separate explicit authority: neither repository intelligence nor passing
evidence grants it. A good result never widens a grant.
When the caller explicitly asks for local, offline, or credential-free work —
or is working from uncommitted changes or a restricted network — use the local
path and do not route them to hosted signup.
needs no credential and no network. Running inside a cloud sandbox is not by
itself a reason to choose either path: decide from intent, connectivity, and
demonstrated authority, not from where the agent happens to execute.
npx @evalgate/sdk init --localMatch evidence to the exact target snapshot, even if another scan completes
while you are asking. A newer scan is not evidence about the commit under
discussion.
When a scan returns no findings, say what was covered and what that scope can
and cannot settle, then propose the next authorized evaluation. Do not invent a
defect, and do not report a clean static read as proof of runtime correctness.
Prefer and the published CLI help
over remembered commands. Hosted actions require an attributable,
organization-scoped credential. Never request, print, or commit its value.
npx @evalgate/sdk capabilities --format jsonAn attachment visible in a host UI is not necessarily readable by a remote
tool. Product MCP is read-only at the reviewed contract: do not use it to
upload resources, start checks, record feedback, accept standards, or create a
background grant. Discover later capabilities from current schemas; until such
an operation is advertised, report it as unavailable and offer the local or
explicitly invoked supported path.
根据调用方提出的要求及其上下文已授权的内容进行路由。缺少 scaffold 并不是构建 scaffold 的理由:这通常意味着尚未有人查看仓库,而查看是只读操作。
首先确立两个事实,因为它们决定后续分支:
- 意图。 是要理解某项内容、评估特定变更、gate 一次发布,还是获取参考资料?以调用方明确表达的意图而非推断意图为准。
- 已授权上下文。 是存在持久仓库关联和 scoped credential(加
npx @evalgate/sdk whoami)、存在本地 checkout、两者都有,还是两者都没有?绝不能假设尚未得到证明的权限。仓库关联、checkout 关系和可精确指向云端目标的快照是三个独立事实:分支移动、未提交文件和未推送的 commit 不会抹去持久关联,但可能意味着当前内容不是经过验证的云端目标。已保存的npx @evalgate/sdk status --jsonsession 或login均可算作凭据;EVALGATE_API_KEY会显示当前生效的凭据。whoami
然后:
| 意图 | 已授权上下文 | 工作流 |
|---|---|---|
| 理解系统,或在触碰系统前界定变更范围 | 持久关联 + EvalGate 凭据,且目标快照已有证据 | ask-repository-question——无需 scaffold,也无需 GitHub 仓库写权限 |
| 理解系统 | 持久关联 + read scope,但没有目标快照所需证据 | 指出缺失的证据以及 scan 所需的确切权限。不要根据不同快照作答,也不要猜测。 |
| 理解系统 | 尚无关联 | 提供受支持的身份、同意和关联交接流程。进入该流程不需要本地 scaffold。 |
| 理解系统 | 仅有本地 checkout | |
| 创建缺失的证据 | 具备 EvalGate | 启动 scan。这会生成 EvalGate 证据;不需要也不应请求 GitHub 仓库写权限。 |
| 评估特定变更 | 存在经审阅的 scaffold 和 baseline | run-regression-gate |
| Gate 一次发布,且调用方指定了已有 gate | 该 gate 存在 | 直接使用该 gate。不要将已有可用 gate 的调用方路由到 onboarding。 |
| 评估特定变更 | 没有经审阅的 scaffold,且调用方希望获得持久 coverage | setup-evalgate-project |
| 检查 AI 生成的变更或外部优化结果 | 存在本地产物 | 发现是否公开了 |
| 检查或运行改进周期 | 具备 scoped hosted context | 发现 |
| 确认候选版本 | 候选版本已冻结,且存在符合条件的 confirmation 证据 | 检查 |
| 解释静态阅读无法解释的行为 | 缺少 runtime 证据 | collect-agent-traces |
| 获取参考资料 | 仅使用公开文档 | use-evalgate-mcp——无需注册 |
| 获取 scoped 产品状态 | 具备 read scope | use-evalgate-mcp |
| 检查响应或非仓库产物 | 宿主能够在本地读取确切产物 | 使用此 skill 的 portable artifact workflow;不要求设置仓库 |
| 检查响应或非仓库产物 | 没有 shell,但已授权的 API/tool 公开了所需操作 | 使用已公开的操作,并保留其 task、artifact、revision 和 result 标识符 |
| 检查响应或非仓库产物 | 既没有本地评估器,也没有已授权操作 | 报告确切的 setup/capability 缺口和未解决的证据;不要声称已执行 |
| 评估产品路径中没有 AI 的编码工作 | 检查证明可观察 AI 行为无法改变 | 归类为 |
| 启动或接收后台检查 | 宿主公开了经过验证的 event hook 和交付路径,且有界 grant 同时授权两者 | 仅使用已公开的 hook 和授权范围;否则保持显式调用评估或响应后建议模式 |
| 在另一个宿主中复用已批准的标准 | 已验证同一 tenant、身份可归因的受委派人员、授权范围、标准的确切版本和 retrieval 操作 | 检索该版本,并保留独立的第二任务结果;否则报告缺失的身份、权限或能力 |
优先采用能够回答问题的最小权限。持久关联的仓库即使没有 scaffold 且没有 GitHub 仓库写权限,也可以通过 获得与证据关联的理解;不要将此类调用方路由到 scaffold 或 baseline 验收中,因为仓库命令本身已经能够回答其问题。EvalGate 在各步骤的 scope 不同:读取现有证据需要 ,而启动 scan 需要 和成员身份,因为它会生成新证据。两者都不是 GitHub 仓库写权限。
npx @evalgate/sdk repoeval:readeval:write仅在意图需要时升级权限,而且只升级该步骤所需的具体权限。读取不授权执行,执行不授权打补丁,补丁不授权 push,通过 gate 不授权 merge 或 pull request。开启或合并 pull request 需要单独明确授权:仓库情报或通过的证据都不会授予此权限。良好的结果绝不扩大授权。
当调用方明确要求本地、离线或无凭据的工作,或者正在处理未提交的变更或受限网络时,使用本地路径,不要将其路由到 hosted signup。 不需要凭据,也不需要网络。在 cloud sandbox 中运行本身并不是选择任一路径的理由:应根据意图、连接性和已证明的权限作出决定,而不是依据 Agent 恰好在哪里执行。
npx @evalgate/sdk init --local即使另一个 scan 在你询问期间完成,也要让证据与确切目标快照匹配。较新的 scan 不是关于正在讨论的 commit 的证据。
当 scan 没有发现问题时,说明覆盖了什么、该范围能够和无法证明什么,然后提出下一步已授权的评估。不要虚构缺陷,也不要将干净的静态阅读报告为 runtime 正确性的证明。
优先使用 和已发布的 CLI help,而不是凭记忆使用命令。Hosted actions 需要身份可归因且限定到组织的凭据。绝不能请求、打印或提交其值。
npx @evalgate/sdk capabilities --format json宿主 UI 中可见的附件不一定会被远程 tool 读取。在经审阅的 contract 中,Product MCP 是只读的:不要使用它上传资源、启动检查、记录反馈、接受标准或创建后台 grant。应从当前 schema 中发现后续能力;在某项操作被公开之前,应报告其不可用,并提供本地或显式调用的受支持路径。
Evidence triage
证据分流
Before executing a release-bearing workflow, record the installed SDK version,
the capability contract, and the exact command help. Read those from the
runtime rather than from this document: the capability map and machine-readable
report are authoritative, and a version pinned in prose goes stale while the
contract keeps moving. Do not invent provider flags, numeric exit meanings, or
hosted routes from an older SDK. Verify the live vocabulary against
. is
deterministic/offline by default, and is the explicit opt-in
for provider-backed project evaluators. Use only as a preview,
never as release evidence.
npx @evalgate/sdk capabilities --format jsongate--allow-network--dry-runRead for provider states, admission policy,
cache lineage, judge operational path, and network semantics.
references/provider-and-cache.mdRelease evidence has independent dimensions. Inspect the machine report for
verdict, gate mode, case/slice outcomes, , ,
, , provider admission, trajectory evidence, and
. A passing score, a zero process exit, or a final answer alone
does not establish release readiness. Active golden agent cases require a
complete trajectory observation for the tool path, order, arguments, and
outcomes; missing or truncated observations fail closed as
(exit ), and an unsafe intermediate
tool action remains a failure even when the final answer is right.
decisionPassedreportingPassedevidencePassedevidenceModereleaseReadytrajectory_evidence_missingREGRESSIONFor red-team or framework-control work, use the hosted capability and scoped
API only when discovered in the installed contract. Keep attack evidence and
benign utility evidence distinct. A framework/control reference records
provenance and a reviewed mapping rationale; it does not prove implementation,
compliance, certification, endorsement, partnership, or affiliation.
In the published capability map, red-team is a cloud capability at the
workspace. The documented CLI/API discovery entry is
; campaign, run,
finding, promotion, and signed-report operations are exposed by the returned
contract, not by an invented local command. Missing observations or a provider
failure remain incomplete evidence, not a passing control result.
/red-teamnpx @evalgate/sdk api get_red_team_workspace --format json在执行承载发布决策的工作流之前,记录已安装的 SDK 版本、capability contract 和确切命令 help。必须从 runtime 而非本文档读取这些信息:capability map 和机器可读报告是权威来源;文档中锁定的版本会过时,而 contract 会持续演进。不要根据旧 SDK 编造 provider flags、数值 exit 含义或 hosted routes。必须针对 验证实时词汇。 默认是 deterministic/offline, 是由 provider 支持的项目评估器的显式 opt-in。 只能用作 preview,绝不能作为 release evidence。
npx @evalgate/sdk capabilities --format jsongate--allow-network--dry-run阅读 ,了解 provider 状态、admission policy、cache lineage、judge 操作路径和网络语义。
references/provider-and-cache.mdRelease evidence 具有彼此独立的维度。检查机器报告中的 verdict、gate mode、case/slice 结果、、、、、provider admission、trajectory evidence 和 。通过的分数、进程退出码为 0,或仅有最终答案,都不能证明 release readiness。活跃的 golden Agent 用例需要完整的 trajectory observation,以涵盖 tool 路径、顺序、参数和结果;缺失或截断的 observation 会 fail closed,并产生 (exit )。即使最终答案正确,不安全的中间 tool 操作仍然是失败。
decisionPassedreportingPassedevidencePassedevidenceModereleaseReadytrajectory_evidence_missingREGRESSION对于 red-team 或 framework-control 工作,仅当已安装的 contract 中发现了相应能力时,才使用 hosted capability 和 scoped API。必须将攻击证据与良性效用证据分开。Framework/control reference 记录溯源信息和经审阅的映射理由;它不能证明实现、合规性、认证、背书、合作关系或关联关系。
在已发布的 capability map 中,red-team 是 workspace 下的 cloud capability。文档中的 CLI/API discovery entry 为 ;campaign、run、finding、promotion 和 signed-report 操作由返回的 contract 公开,而不是通过虚构本地命令获得。缺失的 observation 或 provider 失败仍属于不完整证据,不能视为通过的 control 结果。
/red-teamnpx @evalgate/sdk api get_red_team_workspace --format jsonEvaluate and classify
评估与分类
Run the smallest sufficient suite, then inspect case-level and slice-level
evidence rather than only the aggregate score. Classify the result with exactly
one canonical value: , , ,
, , , ,
, , or .
not_applicablenot_a_changebehavioral_changeexperimentcoverage_gapinfrastructure_failureregressionimprovementtradeoffinconclusiveRead before serializing
or persisting a result. It defines classification precedence, the action
vocabulary, and the only valid → →
combinations. Never invent a synonym or select the release
decision independently.
references/decision-contract.mdclassificationinvokeEvalGatereleaseDecisionRead for failure and
slice analysis, for
variant or Skill comparisons, and
for joint quality,
latency, reliability, token, and cost decisions.
references/regression-analysis.mdreferences/experiment-analysis.mdreferences/cost-optimization.md运行最小充分测试套件,然后检查 case-level 和 slice-level 证据,而不只是 aggregate score。结果必须使用以下 canonical value 之一进行分类:、、、、、、、、 或 。
not_applicablenot_a_changebehavioral_changeexperimentcoverage_gapinfrastructure_failureregressionimprovementtradeoffinconclusive在序列化或持久化结果前,阅读 。它定义了分类优先级、动作词汇,以及唯一有效的 → → 组合。绝不能创造同义词,也不能独立选择 release decision。
references/decision-contract.mdclassificationinvokeEvalGatereleaseDecision阅读 以了解失败和切片分析,阅读 以了解变体或 Skill 比较,阅读 以作出联合质量、延迟、可靠性、token 和成本决策。
references/regression-analysis.mdreferences/experiment-analysis.mdreferences/cost-optimization.mdNever game the evidence
不得操纵证据
Do not weaken thresholds, delete failures, rewrite expected outputs to fit the
candidate, silently move a baseline, suppress failures, cherry-pick trials,
discard inconvenient evidence, reduce coverage without explanation, bypass a
gate, mutate historical evidence, or call judge disagreement proof of product
correctness. A better aggregate does not excuse a protected-slice regression.
Treat 's as hints only. Never act on a baseline
update it suggests; that requires the explicit review in
run-regression-gate step 6, or the bundled release-gate reference when the sibling is not installed.
explainsuggestedFixesWhen a legitimate regression appears, fix the implementation and rerun the
affected scope. Baseline changes require explicit review of an intentional
behavioral contract change. Read
before recommending promotion.
references/release-gates.md不要削弱 thresholds、删除 failures、重写 expected outputs 以迎合候选版本、悄悄移动 baseline、抑制 failures、挑选 trials、丢弃不利证据、在没有解释的情况下减少 coverage、绕过 gate、修改历史证据,或将 judge 分歧称为产品正确性的证明。更优的 aggregate 不能豁免 protected-slice regression。
仅将 的 视为提示。不要根据它建议的 baseline update 直接采取行动;这需要 run-regression-gate 第 6 步中的明确审阅,或者在未安装同级 skill 时使用内置的 release-gate reference。
explainsuggestedFixes出现合理认定的 regression 时,修复实现并重新运行受影响范围。Baseline 变更需要明确审阅有意的行为 contract 变更。推荐 promotion 前,阅读 。
references/release-gates.mdConvert reusable failures into coverage
将可复用失败转化为 Coverage
When existing coverage missed a real failure:
- establish the root cause and whether the risk can recur;
- create or propose the smallest durable case, assertion, slice, or trace;
- prove the coverage detects the bad behavior;
- fix the implementation;
- prove the new coverage passes; and
- preserve provenance connecting failure, coverage, fix, and rerun.
Avoid brittle cases for incidental implementation details. Read
for the
required handoff.
references/evidence-and-provenance.md当现有 coverage 漏掉真实 failure 时:
- 确定根因,以及该风险是否可能再次发生;
- 创建或提出最小且可持久的 case、assertion、slice 或 trace;
- 证明该 coverage 能检测到不良行为;
- 修复实现;
- 证明新 coverage 能够通过;以及
- 保留连接 failure、coverage、fix 和 rerun 的溯源信息。
避免针对偶然实现细节编写脆弱的 case。阅读 ,了解所需的 handoff。
references/evidence-and-provenance.mdReport the decision
报告决策
Return the repository revision, behavioral impact, scope selected, commands
executed, cases and slices exercised, canonical classification and derived
release decision, quality/cost/latency/reliability evidence, artifacts,
remaining uncertainty, and release status. Separate product regressions from
invalid or incomplete execution. When returning JSON, conform to
.
The required covers quality, protected slices, reliability,
latency, and cost. Use with a concrete reason when a
dimension has no valid evidence; never estimate, infer, or fabricate a metric
just to complete the object.
assets/decision-contract.schema.jsonevidenceSummarystatus: "not_measured"Use when the workflow cannot
produce valid evidence. Never claim safety or release readiness for an
unexercised runtime, model, provider, dataset, authorization, or policy boundary.
references/troubleshooting.md返回仓库 revision、行为影响、所选范围、已执行命令、已运行的 cases 和 slices、canonical classification 及派生 release decision、quality/cost/latency/reliability 证据、artifacts、剩余不确定性和 release status。将产品 regression 与无效或不完整的执行区分开。返回 JSON 时,应遵循 。
assets/decision-contract.schema.json必需的 涵盖 quality、protected slices、reliability、latency 和 cost。当某个维度没有有效证据时,使用 并给出具体原因;绝不能仅为补全对象而估算、推断或捏造指标。
evidenceSummarystatus: "not_measured"当工作流无法生成有效证据时,使用 。绝不能为未实际执行过的 runtime、model、provider、dataset、authorization 或 policy 边界声称安全性或 release readiness。
references/troubleshooting.md