evals-init
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
Chineseevals-init
evals-init
What this skill does
这个 skill 的作用
Initialize the project-level evaluation directory structure following EDD (Eval-Driven Development) principles to prepare for systematic evaluation development. This is completely standalone with zero spec-kit dependencies.
Output:
- Directory Structure - with proper organization (promptfoo | deepeval)
evals/{system}/ - Security Baseline - Auto-created graders for PII leakage, prompt injection, hallucination detection, misinformation detection
- Configuration Files - Standalone config.yml and goldset templates under
.adlc/evals/ - Auto-handoff to to begin error analysis
/evals-specify
Key EDD Principles Applied:
- Principle I: Spec-Driven Contracts - Evals validate spec compliance
- Principle II: Binary Pass/Fail - No Likert scales in grader templates
- Principle IV: Evaluation Pyramid - Tier 1 (fast) + Tier 2 (goldset) structure
- Principle IX: Test Data as Code - Version control setup for datasets
按照 EDD(Eval-Driven Development)原则初始化项目级评估目录结构,为系统化评估开发做好准备。此过程完全独立,不依赖 spec-kit。
输出:
- 目录结构 - ,采用适当的组织方式(promptfoo | deepeval)
evals/{system}/ - 安全基线 - 自动创建用于 PII leakage、prompt injection、hallucination detection 和 misinformation detection 的 graders
- 配置文件 - 在 下提供独立的 config.yml 和 goldset 模板
.adlc/evals/ - 自动移交至 ,开始错误分析
/evals-specify
所应用的 EDD 关键原则:
- 原则 I:Spec-Driven Contracts - Evals 用于验证 spec 符合性
- 原则 II:Binary Pass/Fail - grader 模板中不使用 Likert 量表
- 原则 IV:Evaluation Pyramid - Tier 1(快速)+ Tier 2(goldset)结构
- 原则 IX:Test Data as Code - 为数据集设置版本控制
When to use
何时使用
- Starting systematic evaluation: Set up the initial evaluation harness for your application
- EDD Adoption: Converting from traditional testing to evaluation-driven development
- Security-first evaluation: Auto-generate baseline security checks from the start
- 开始系统性评估:为应用搭建初始 evaluation harness
- 采用 EDD:从传统 testing 转向 evaluation-driven development
- 安全优先评估:从一开始自动生成 baseline security checks
When NOT to use
何时不应使用
- Evals directory already exists: Use to run tests, or
/evals-validateto add criteria/evals-specify - Evaluating team directives: This is for project-level application behavior testing, not directives compliance
- Evals 目录已存在:使用 运行测试,或使用
/evals-validate添加评估标准/evals-specify - 评估团队 directives:此 skill 用于项目级应用行为测试,而非 directives 合规性检查
Process
流程
User Input
用户输入
text
$ARGUMENTSParse flags from the arguments first, then treat remaining text as focus areas:
- — Choose
--system SYSTEMorpromptfoo. If omitted, choose interactively based on tech stack.deepeval - Remaining text — System description (focus setup)
text
$ARGUMENTS先解析参数中的 flags,再将其余文本视为重点范围:
- — 选择
--system SYSTEM或promptfoo。如果未指定,则根据技术栈交互式选择。deepeval - 其余文本 — 系统描述(设置重点)
Execution Steps
执行步骤
Phase 1: Tech Stack Detection
阶段 1:技术栈检测
- Scan project manifests (,
package.json,requirements.txt,Cargo.toml, etc.)go.mod - Recommends PromptFoo for mixed/JS stacks; DeepEval for Python-native stacks
- 扫描项目清单文件(、
package.json、requirements.txt、Cargo.toml等)go.mod - 对混合/JS 技术栈推荐 PromptFoo;对 Python 原生技术栈推荐 DeepEval
Phase 2: Create Directory Structure
阶段 2:创建目录结构
Creates:
evals/
├── {system}/ # promptfoo | deepeval
│ ├── goldset.md # Published goldset
│ ├── goldset.json # Auto-generated for system consumption
│ ├── config.yml # System-specific configuration
│ ├── config.{js,py} # Generated system config (.js for promptfoo, .py for deepeval)
│ └── graders/ # Binary pass/fail graders
│ ├── check_pii_leakage.py # Security baseline
│ ├── check_prompt_injection.py # Security baseline
│ ├── check_hallucination.py # Security baseline
│ └── check_misinformation.py # Security baseline
├── results/ # Git-ignored run outputs
└── .adlc/
└── drafts/evals/ # Draft eval records (Markdown + YAML)创建:
evals/
├── {system}/ # promptfoo | deepeval
│ ├── goldset.md # Published goldset
│ ├── goldset.json # Auto-generated for system consumption
│ ├── config.yml # System-specific configuration
│ ├── config.{js,py} # Generated system config (.js for promptfoo, .py for deepeval)
│ └── graders/ # Binary pass/fail graders
│ ├── check_pii_leakage.py # Security baseline
│ ├── check_prompt_injection.py # Security baseline
│ ├── check_hallucination.py # Security baseline
│ └── check_misinformation.py # Security baseline
├── results/ # Git-ignored run outputs
└── .adlc/
└── drafts/evals/ # Draft eval records (Markdown + YAML)Phase 3: Configuration Copy
阶段 3:复制配置
- Create if missing.
.adlc/evals/ - Copy to
skills/evals/evals-templates/evals-config-template.yml..adlc/evals/evals-config.yml
- 如果缺失,则创建 。
.adlc/evals/ - 将 复制到
skills/evals/evals-templates/evals-config-template.yml。.adlc/evals/evals-config.yml
Phase 4: Auto-Handoff
阶段 4:自动移交
Trigger to begin error analysis.
/evals-specify触发 ,开始错误分析。
/evals-specifyVerification
验证
- exists (initially empty)
evals/{system}/goldset.md - exists
.adlc/evals/evals-config.yml - Graders directory populated with 4 security baseline python scripts
- Results directory contains to prevent versioning traces
.gitignore - Handover report generated with recommended framework and next steps
- 存在(初始为空)
evals/{system}/goldset.md - 存在
.adlc/evals/evals-config.yml - graders 目录中有 4 个安全基线 Python 脚本
- results 目录包含 ,防止运行轨迹被纳入版本控制
.gitignore - 生成移交报告,其中包含推荐框架和后续步骤