Back to Details

skill-eval-scaffolder

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Skill Eval Scaffolder

Skill评估模板生成工具

Create minimal, schema-valid eval manifests for repository skills.
Scope: Eval scaffolding only. Do not run live behavioral evals unless the maintainer explicitly approves.
为仓库Skill生成符合规范的极简评估清单模板。
**适用范围：**仅用于评估模板搭建。除非维护者明确批准，否则请勿运行实时行为评估。

Dispatch

命令参数

$ARGUMENTSAction
<name>
Preview scaffold for 
skills/<name>/evals/evals.json
<name> --apply
Write scaffold when file is missing or user approved overwrite
check <name>
Validate an existing eval manifest
EmptyShow usage gallery and manifest schema summary
参数操作
<name>
预览
skills/<name>/evals/evals.json
的模板
<name> --apply
当文件不存在或用户批准覆盖时写入模板
check <name>
验证现有评估清单
无参数显示使用示例和清单规范摘要

Workflow

工作流程

  1. Confirm 
    skills/<name>/SKILL.md
    exists.
  2. Run the bundled scaffold script:
bash
uv run python skills/skill-eval-scaffolder/scripts/scaffold_evals.py <name>
  1. Add explicit, implicit, and negative-control cases before claiming adequate coverage.
  2. Validate with the bundled check script.
  1. 确认
    skills/<name>/SKILL.md
    已存在。
  2. 运行内置的模板生成脚本：
bash
uv run python skills/skill-eval-scaffolder/scripts/scaffold_evals.py <name>
  1. 添加显式、隐式和负面控制用例，之后才能认为覆盖足够。
  2. 使用内置的检查脚本验证。

Manifest Shape

清单结构

Each 
evals/evals.json
uses:
  • skill_name
    — kebab-case directory name
  • evals
    — non-empty list of cases with 
    id
    , 
    prompt
    , 
    expected_output
    , optional 
    files
    , 
    assertions
每个
evals/evals.json
包含：
  • skill_name
    — 短横线命名法（kebab-case）的目录名称
  • evals
    — 非空的用例列表，每个用例包含
    id
    、
    prompt
    、
    expected_output
    ，可选
    files
    、
    assertions

Validation Contract

验证协议

bash
uv run python skills/skill-eval-scaffolder/scripts/check.py
bash
uv run python skills/skill-eval-scaffolder/scripts/check.py

Critical Rules

关键规则

  1. Never overwrite an existing eval manifest without explicit user approval.
  2. Include at least one explicit-invocation case in new scaffolds.
  3. Run 
    validate_evals.py
    before declaring the scaffold complete.
  4. Route live eval execution to skill-creator 
    eval
    mode or maintainer CI workflows.
  5. Keep prompts grounded in the skill dispatch table and NOT-for boundaries.
  1. 未经用户明确批准，绝不能覆盖现有评估清单。
  2. 新模板中至少包含一个显式调用用例。
  3. 在宣布模板完成前运行
    validate_evals.py
    。
  4. 将实时评估执行路由到Skill创建者的
    eval
    模式或维护者CI工作流。
  5. 提示语需基于Skill调度表和禁止边界设定。