skill-audit

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Skill Audit

Purpose

目的

Decide whether a skill is fit to ship. Structure is not the question — scripts answer that faster and more reliably. This skill owns the five questions a script cannot answer:

Will the description ever fire, and does it fire on the right situations?
Can an agent follow the instructions to the end without guessing?
Does the body earn the tokens it costs in every session?
Does
```
compatibility
```
match what the body actually requires?
Can this skill damage something without being told to?

判断某个Skill是否适合发布。结构问题不在此范畴——脚本可以更快、更可靠地解决这类问题。本Skill负责处理脚本无法回答的五个问题：

描述是否会触发？是否在正确的场景下触发？
Agent能否无需猜测即可完整执行指令？
主体内容在每次会话中消耗的token是否物有所值？
```
compatibility
```
是否与主体实际需求匹配？
该Skill是否可能在未被指令的情况下造成损害？

When to Use This Skill

何时使用本Skill

Before committing a change to any skill directory — this is the acceptance gate
"Audit skills", "score this skill", "check compliance with the spec"
"Is this skill Codex-ready?"
"Why does this skill never get invoked?"
Reviewing a PR that touches
```
SKILL.md
```
,
```
references/
```
, or
```
agents/openai.yaml
```

在提交对任意Skill目录的变更之前——这是验收关卡
场景包括：“审核Skills”、“为该Skill评分”、“检查是否符合规范”
“该Skill是否已准备好接入Codex？”
“为什么这个Skill从未被调用？”
审查涉及
```
SKILL.md
```
、
```
references/
```
或
```
agents/openai.yaml
```
的PR

Division of Labour

分工

Anything mechanically checkable belongs to a script. Never re-derive by hand what one of these already answers, and never score a skill down for something a script reports green.

Owner	Answers	Failure mode
`.github/scripts/validate-skills.sh`	Marketplace and plugin manifests; canonical `<group>/skills/<name>` real directory; required frontmatter; `name` matches directory and format; description ≤ 1024 and discovery entry ≤ 1536; `SKILL.md` referencing bundled files it does not ship	Hard, fails CI
`scripts/validate-codex.sh`	Codex catalog membership, `compatibility` ↔ catalog agreement, `agents/openai.yaml` presence, Codex ⊆ OpenCode ∩ Pi host subset rule	Hard, fails CI
`scripts/skill-metrics.mjs`	Body size and token estimate, largest inline block, unreachable bundled files, nested reference chains, untagged fences, heading skips, host-mechanism hits, `AskUserQuestion` fallback wording, cross-skill discovery overlap	Advisory measurements
This skill	The five judgment questions above	Scored 1–5

Requires Node (for

scripts/skill-metrics.mjs

) and

jq

(for the two repo validators). When a validator is absent — auditing a repo that is not

edloidas/skills

— say so and note reduced confidence rather than reimplementing it.

所有可通过机械方式检查的内容都由脚本负责。切勿手动推导脚本已能回答的问题，也切勿因脚本已判定为合格的内容而给Skill扣分。

负责方	解决的问题	失败模式
`.github/scripts/validate-skills.sh`	市场和插件清单；标准 `<group>/skills/<name>` 实际目录；必填前置内容； `name` 与目录及格式匹配；描述≤1024字符、发现条目≤1536字符； `SKILL.md` 引用了未打包的文件	硬性失败，导致CI失败
`scripts/validate-codex.sh`	Codex目录成员资格、 `compatibility` 与目录的一致性、 `agents/openai.yaml` 是否存在、Codex ⊆ OpenCode ∩ Pi主机子集规则	硬性失败，导致CI失败
`scripts/skill-metrics.mjs`	主体内容大小和token估算、最大内联块、无法访问的打包文件、嵌套引用链、未标记的代码块、标题跳跃、主机机制命中、 `AskUserQuestion` 回退措辞、跨Skill发现重叠	参考性测量指标
本Skill	上述五个判定类问题	1-5分评分

需要Node（运行

scripts/skill-metrics.mjs

）和

jq

（运行两个仓库验证器）。当验证器缺失时——比如审核的仓库不是

edloidas/skills

——需说明情况并标注可信度降低，而非重新实现验证器。

Modes

模式

Invocation	Scope
`/skill-audit`	Skills changed on this branch; falls back to all when nothing changed
`/skill-audit changed`	Skills changed on this branch only
`/skill-audit all`	Every skill, excluding `skill-audit` itself
`/skill-audit <name> [<name> ...]`	Named skills, including `skill-audit` when named

调用方式	范围
`/skill-audit`	本分支中变更的Skills；若无变更则默认审核全部
`/skill-audit changed`	仅审核本分支中变更的Skills
`/skill-audit all`	审核所有Skill，不包括 `skill-audit` 本身
`/skill-audit <name> [<name> ...]`	指定名称的Skills，若指定则包括 `skill-audit`

Workflow

工作流程

Step 1: Resolve scope

步骤1：确定范围

For

changed

mode, list skill directories touched by the branch, working tree included:

bash

BASE=${BASE:-$(git symbolic-ref --quiet --short refs/remotes/origin/HEAD 2>/dev/null || echo origin/HEAD)}
{ git diff --name-only "$(git merge-base HEAD "$BASE")"...HEAD; git status --porcelain | awk '{print $NF}'; } \
  | grep -oE '^[a-z]+/skills/[a-z0-9-]+' | sort -u

BASE

defaults to the repository's default branch. Set it explicitly when the branch forked from somewhere else —

BASE=epic-6

under an epic — or the scope picks up the whole epic.

A deleted skill directory appears here and cannot be audited — report it as removed and move on. If the scope resolves to nothing, say "No skills to audit" and stop.

For named skills, resolve each argument against

<group>/skills/<name>/SKILL.md

. If one does not resolve, list the available skill paths and stop rather than guessing.

对于

changed

模式，列出分支中修改的Skill目录（包括工作区内容）：

bash

BASE=${BASE:-$(git symbolic-ref --quiet --short refs/remotes/origin/HEAD 2>/dev/null || echo origin/HEAD)}
{ git diff --name-only "$(git merge-base HEAD "$BASE")"...HEAD; git status --porcelain | awk '{print $NF}'; } \
  | grep -oE '^[a-z]+/skills/[a-z0-9-]+' | sort -u

BASE

默认是仓库的默认分支。当分支从其他分支分叉时——比如从epic分支

epic-6

分叉——需显式设置

BASE=epic-6

，否则范围会包含整个epic分支的内容。

已删除的Skill目录会出现在此处，无法审核——需报告其已移除并继续。若范围解析为空，则提示“无Skill可审核”并停止。

对于指定名称的Skills，将每个参数与

<group>/skills/<name>/SKILL.md

进行匹配。若某个参数无法匹配，则列出可用的Skill路径并停止，而非猜测。

Step 2: Run the structural gate

步骤2：运行结构验证关卡

Run both validators from the repo root, once, before auditing anything:

bash

bash .github/scripts/validate-skills.sh
bash scripts/validate-codex.sh

Keep the exact

::error::

lines. Every error naming a skill in scope belongs in that skill's report; the rest go under cross-cutting issues. A validator that cannot run — missing

jq

, missing file — is recorded as such, not treated as a pass.

Report only. Never run

scripts/skills-packaging.sh sync-repo

during an audit; a stale generated layer is a finding, not something to quietly fix.

在仓库根目录运行两个验证器一次，再开始审核：

bash

bash .github/scripts/validate-skills.sh
bash scripts/validate-codex.sh

保留准确的

::error::

行。所有涉及范围内Skill的错误都应归入该Skill的报告；其余错误归入跨领域问题。若验证器无法运行——缺失

jq

、缺失文件——需记录该情况，而非视为通过。

仅做报告。审核期间切勿运行

scripts/skills-packaging.sh sync-repo

；过时的生成层是一个发现项，而非需要悄悄修复的内容。

Step 3: Collect measurements

步骤3：收集测量数据

bash

node <skill-dir>/scripts/skill-metrics.mjs --only <skill-path> [<skill-path> ...]

Omit

--only

for all-skills mode. The discovery-overlap section at the end compares every skill in the run against every other, so run it across the whole set at least once when scoring Discovery — a pair only shows up when both halves are present.

bash

node <skill-dir>/scripts/skill-metrics.mjs --only <skill-path> [<skill-path> ...]

全Skill模式下省略

--only

参数。末尾的发现重叠部分会将运行中的每个Skill与其他所有Skill进行比较，因此在为发现能力评分时，至少要针对整个集合运行一次——只有当两个Skill都存在时，配对才会显示。

Step 4: Score judgment, one skill at a time

步骤4：逐一为判定维度打分

Read

references/subagent-prompt.md

and dispatch one cheap, read-only worker per skill, launching them all before waiting on any result. Replace

{{SKILL_PATH}}

{{REPO_ROOT}}

, and

{{METRICS}}

(that skill's block from Step 3) in the template. Each worker reads the skill's files and returns the structured block the template specifies.

If the host has no facility for spawning workers, run the same prompt inline, one skill at a time. If a worker fails or returns unparseable output, mark that skill Audit Incomplete with the reason and continue.

references/evaluation-rubric.md

holds the full criteria and scoring anchors. It is the reference for resolving a borderline score — it is not injected into workers, which carry a condensed copy already.

读取

references/subagent-prompt.md

，为每个Skill分配一个低成本、只读的工作进程，在等待结果前启动所有进程。替换模板中的

{{SKILL_PATH}}

、

{{REPO_ROOT}}

和

{{METRICS}}

（步骤3中该Skill的测量块）。每个工作进程读取Skill的文件并返回模板指定的结构化块。

若主机不支持生成工作进程，则逐个Skill内联运行相同的提示。若工作进程失败或返回无法解析的输出，则标记该Skill为审核未完成并注明原因，继续后续操作。

references/evaluation-rubric.md

包含完整的标准和评分基准。它是解决边界分数问题的参考——不会注入到工作进程中，工作进程已携带简化版本。

Step 5: Verdict

步骤5：结论

Reject any score that arrives without cited evidence and re-run that skill. Then per skill:

Verdict	Bar
PASS	No structural errors, and every category ≥ 4
PASS WITH NOTES	No structural errors, and the lowest category is 3
FAIL	Any structural error, or any category ≤ 2

Overall is the mean of the five categories to one decimal. Report the minimum next to it — a skill at 4.4 overall with a 2 in Safety is a FAIL, and the average must never hide that.

拒绝任何无引用证据的评分，并重新审核该Skill。然后针对每个Skill：

结论	标准
PASS	无结构错误，且所有维度得分≥4
PASS WITH NOTES	无结构错误，最低维度得分为3
FAIL	存在任何结构错误，或任何维度得分≤2

整体得分是五个维度的平均值，保留一位小数。同时报告最低分——比如整体得分4.4但安全维度得分为2的Skill仍判定为FAIL，平均值绝不能掩盖这一点。

Report Format

报告格式

All-skills and

changed

mode open with a summary, sorted worst first:

markdown

undefined

全Skill模式和

changed

模式以摘要开头，按最差结果排序：

markdown

undefined

Skill Audit — <scope>

Audited: N | Average: X.X / 5 | Failing: N

Skill	Discovery	Instructions	Context	Portability	Safety	Overall	Min	Verdict
path	X	X	X	X	X	X.X	X	PASS


Then, when they apply:

```markdown

审核数量: N | 平均分: X.X / 5 | 不合格数量: N

Skill	发现能力	指令质量	内容效率	可移植性	安全性	总分	最低分	结论
路径	X	X	X	X	X	X.X	X	PASS


然后添加适用的内容：

```markdown

Structural Errors

结构错误

```
::error::
```
line verbatim — affected skill

```
::error::
```
行原文 — 受影响的Skill

Cross-Cutting Issues

跨领域问题

Description (affects N skills)

描述内容（影响N个Skill）

Top Recommendations

首要建议

Actionable recommendation


Include a full breakdown for every named skill in single-skill mode, and in the other
modes for any skill that is not a clean PASS:

```markdown

可执行的建议


单Skill模式下需包含每个指定Skill的完整细分内容；其他模式下需包含所有非完全PASS的Skill的完整细分：

```markdown

<group>/skills/<name> — X.X / 5 — <verdict>

<group>/skills/<name> — X.X / 5 — <结论>

Category	Score	Evidence
Discovery	X	...
Instruction Quality	X	...
Context Efficiency	X	...
Portability & Integration	X	...
Safety & Robustness	X	...

Issues:

[Category] Description — file:line or quote

Strengths:

Description

Recommendations:

Description

undefined

维度	得分	证据
发现能力	X	...
指令质量	X	...
内容效率	X	...
可移植性与集成	X	...
安全性与鲁棒性	X	...

问题:

[维度] 描述 — 文件:行号或引用内容

优势:

描述内容

建议:

描述内容

undefined

Edge Cases

边缘情况

Skill with only
SKILL.md
— valid. Bundled directories are optional.
Skill directory in scope but deleted on this branch — report as removed, do not score.
Body over 500 lines — a Context Efficiency finding, not a reason to stop reading it, unless the skill carries a budgeted allowance in
```
BODY_LINE_BUDGETS
```
in
```
.github/scripts/validate-skills.sh
```
and is inside it. That is a sanctioned size, already enforced mechanically; scoring it again would report the same finding forever.
Binary files under
assets/
— count them, do not read them.
A validator cannot run — record the reason and say confidence is reduced for whatever that validator covers. Do not substitute a guess for its verdict.
Generated wrapper layer looks stale — a cross-cutting finding whose fix path is
```
scripts/codex/catalog.json
```
plus
```
scripts/skills-packaging.sh sync-repo
```
.
Auditing a repo with no Codex layer — score Portability from
```
compatibility
```
and the body alone; there is no catalog contract to check.
Skills sharing a discovery neighbourhood — a high overlap score is evidence for a Discovery finding against both skills, not just the newer one.

仅包含
SKILL.md
的Skill — 合法。打包目录为可选。
范围内的Skill目录在本分支中已删除 — 报告为已移除，不打分。
主体内容超过500行 — 属于内容效率发现项，但并非停止审核的理由，除非该Skill在
```
.github/scripts/validate-skills.sh
```
的
```
BODY_LINE_BUDGETS
```
中有预算额度且未超出。该大小是经批准的，已由机械方式强制执行；再次打分会重复报告同一发现项。
assets/
下的二进制文件 — 统计数量，无需读取内容。
验证器无法运行 — 记录原因，并说明该验证器负责的内容可信度降低。切勿用猜测替代其结论。
生成的包装层看起来过时 — 属于跨领域发现项，修复路径为
```
scripts/codex/catalog.json
```
加上
```
scripts/skills-packaging.sh sync-repo
```
。
审核无Codex层的仓库 — 仅根据
```
compatibility
```
和主体内容为可移植性打分；无目录合约可检查。
共享发现领域的Skills — 高重叠分数是针对两个Skill的发现能力发现项的证据，而非仅针对较新的Skill。

Rules

规则

Scripts first. Run the validators and the metrics script before forming any opinion.
Evidence or nothing. Every score cites a line, a quote, or a path.
Report only. Never modify a skill, a manifest, or a generated tree during an audit.
Judgment only. Do not score what the structural gate already covers; cite its result.
No fluent-nonsense credit. A polished body that never activates still fails Discovery.
Worst first. Sort by lowest score and lead with what fails.

脚本优先。在形成任何意见前，先运行验证器和指标脚本。
有证据支撑。每个得分都需引用行号、引用内容或路径。
仅做报告。审核期间切勿修改Skill、清单或生成的目录树。
仅做判定。切勿为结构验证关卡已覆盖的内容打分；引用其结果即可。
不认可流畅的无意义内容。表述精良但从未触发的主体内容仍会在发现能力维度失败。
最差优先。按最低分排序，优先展示不合格的内容。