evaluation

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Evaluation Methods for Agent Systems

Agent 系统评估方法

Evaluate agent systems differently from traditional software because agents make dynamic decisions, are non-deterministic between runs, and often lack single correct answers. Build evaluation frameworks that account for these characteristics, provide actionable feedback, catch regressions, and validate that context engineering choices achieve intended effects.

评估 Agent 系统时，应采用不同于传统软件的方式，因为 Agent 会进行动态决策，在不同运行之间具有非确定性，而且通常不存在唯一正确答案。构建评估框架时，应考虑这些特征，提供可操作的反馈、捕获回归问题，并验证 context engineering 选择是否实现了预期效果。

When to Activate

何时启用

Activate this skill when:

Testing agent performance systematically
Validating context engineering choices
Measuring improvements over time
Catching regressions before deployment
Building quality gates for agent pipelines
Comparing different agent configurations
Evaluating production systems continuously

Do not activate this skill for adjacent work owned by other skills:

Designing the LLM judge itself, pairwise comparison, judge calibration, or bias mitigation:
```
advanced-evaluation
```
.
Designing autonomous control surfaces, novelty gates, rollback, or PR approval boundaries:
```
harness-engineering
```
.
Debugging a specific context failure mode before measuring it:
```
context-degradation
```
.

在以下情况下启用此 skill：

系统测试 Agent 性能
验证 context engineering 选择
衡量随时间推移的改进效果
在部署前捕获回归问题
为 Agent 流程构建质量门禁
比较不同的 Agent 配置
持续评估生产系统

对于由其他 skill 负责的相关工作，请勿启用此 skill：

设计 LLM judge 本身、成对比较、judge 校准或偏差缓解：
```
advanced-evaluation
```
。
设计 autonomous control surfaces、novelty gates、rollback 或 PR 审批边界：
```
harness-engineering
```
。
在测量之前调试特定的 context 失败模式：
```
context-degradation
```
。

Core Concepts

核心概念

Focus evaluation on outcomes rather than execution paths, because agents may find alternative valid routes to goals. Judge whether the agent achieves the right outcome via a reasonable process, not whether it followed a specific sequence of steps.

Use multi-dimensional rubrics instead of single scores because one number hides critical failures in specific dimensions. Capture factual accuracy, completeness, citation accuracy, source quality, and tool efficiency as separate dimensions, then weight them for the use case.

Use model-judged evaluation only after deterministic checks and rubrics are stable. When the work centers on judge prompts, pairwise comparison, calibration, or bias mitigation, switch to Advanced Evaluation.

Run deterministic validation before LLM judgment whenever the artifact has machine-checkable structure. Schema validity, duplicate keys, rubric math, manifest sync, retrieval status, and required evidence paths should fail fast before an evaluator spends tokens or returns a subjective score.

Performance Drivers

Apply browsing-agent research when designing evaluation budgets: token usage, tool calls, and model choice can dominate measured performance variance (claim-evaluation-browsecomp-variance).

Factor	Variance Explained	Implication
Token usage	Primary driver	More exploration can improve performance until cost or context quality collapses
Number of tool calls	Secondary driver	More tool use helps only when calls retrieve useful evidence
Model choice	Secondary but multiplicative	Better models often use tokens and tools more efficiently

Act on these implications when designing evaluations:

Set realistic token budgets: Evaluate agents with production-realistic token limits, not unlimited resources.
Compare model upgrades against token increases: Better models may use tokens more efficiently than weaker models with larger budgets.
Validate multi-agent architectures: Extra agents add tokens and tool calls; evaluate them against single-agent baselines.

评估应聚焦于结果而非执行路径，因为 Agent 可能找到其他有效路径来达成目标。判断 Agent 是否通过合理过程实现了正确结果，而不是是否遵循了特定步骤顺序。

不要依赖单一分数，而应使用多维 rubric，因为一个数字会掩盖特定维度中的严重失败。将事实准确性、完整性、引用准确性、来源质量和工具效率作为独立维度分别评估，然后根据具体用例进行加权。

仅在 deterministic checks 和 rubric 已稳定后，才使用 model-judged evaluation。如果工作重点是 judge prompts、成对比较、校准或偏差缓解，请切换到 Advanced Evaluation。

只要产物具有可由机器检查的结构，就应在 LLM judgment 之前运行 deterministic validation。Schema 有效性、重复键、rubric 数学计算、manifest 同步、检索状态和必需证据路径都应快速失败，以免评估器消耗 token 或返回主观分数。

性能驱动因素

设计评估预算时，应应用 browsing-agent 研究结论：token 使用量、工具调用次数和模型选择可能主导所测得的性能方差（claim-evaluation-browsecomp-variance）。

因素	可解释的方差	含义
Token 使用量	主要驱动因素	更多探索可以提升性能，直到成本或 context 质量出现崩溃
工具调用次数	次要驱动因素	只有当调用能检索到有用证据时，更多工具使用才会有所帮助
模型选择	次要但具有乘数效应	更好的模型通常能更高效地使用 token 和工具

设计评估时，应根据这些含义采取行动：

设置现实的 token 预算：使用符合生产实际情况的 token 限制评估 Agent，而不是提供无限资源。
将模型升级与 token 增加进行对比：更好的模型可能比预算更大的较弱模型更高效地使用 token。
验证 multi-agent 架构：额外的 Agent 会增加 token 和工具调用次数；应将其与 single-agent 基线进行对比。

Detailed Topics

详细主题

Evaluation Challenges

评估挑战

Handle Non-Determinism and Multiple Valid Paths

Design evaluations that tolerate path variation because agents may take completely different valid paths to reach goals. One agent might search three sources while another searches ten; both may produce correct answers. Avoid checking for specific steps. Instead, define outcome criteria (correctness, completeness, quality) and score against those, treating the execution path as informational rather than evaluative.

Test Context-Dependent Failures

Evaluate across a range of complexity levels and interaction lengths because agent failures often depend on context in subtle ways. An agent might succeed on simple queries but fail on complex ones, work well with one tool set but fail with another, or degrade after extended interaction as context accumulates. Include simple, medium, complex, and very complex test cases to surface these patterns.

Score Composite Quality Dimensions Separately

Break agent quality into separate dimensions (factual accuracy, completeness, coherence, tool efficiency, process quality) and score each independently because an agent might score high on accuracy but low on efficiency, or vice versa. Then compute weighted aggregates tuned to use-case priorities. This approach reveals which dimensions need improvement rather than averaging away the signal.

处理非确定性与多条有效路径

设计可容忍路径差异的评估，因为 Agent 可能采用完全不同的有效路径达成目标。一个 Agent 可能搜索三个来源，而另一个搜索十个；两者都可能给出正确答案。不要检查具体步骤。相反，应定义结果标准（正确性、完整性、质量）并据此评分，将执行路径视为参考信息，而非评估依据。

测试依赖 Context 的失败

应在不同复杂度级别和交互长度下进行评估，因为 Agent 失败往往以细微方式依赖 context。某个 Agent 可能在简单查询上成功，却在复杂查询上失败；可能使用一组工具时表现良好，但使用另一组工具时失败；也可能在长时间交互、context 不断累积后出现退化。应纳入简单、中等、复杂和非常复杂的测试用例，以发现这些模式。

分别对复合质量维度评分

将 Agent 质量拆分为多个独立维度（事实准确性、完整性、连贯性、工具效率、流程质量）并分别评分，因为 Agent 可能在准确性方面得分很高，但效率得分很低，反之亦然。然后根据用例优先级计算加权汇总分数。这种方法能够揭示需要改进的维度，而不是通过平均计算掩盖具体信号。

Evaluation Rubric Design

评估 Rubric 设计

Build Multi-Dimensional Rubrics

Define rubrics covering key dimensions with descriptive levels from excellent to failed. Include these core dimensions and adapt weights per use case:

Factual accuracy: Claims match ground truth (weight heavily for knowledge tasks)
Completeness: Output covers requested aspects (weight heavily for research tasks)
Citation accuracy: Citations match claimed sources (weight for trust-sensitive contexts)
Source quality: Uses appropriate primary sources (weight for authoritative outputs)
Tool efficiency: Uses right tools a reasonable number of times (weight for cost-sensitive systems)

Convert Rubrics to Numeric Scores

Map dimension assessments to numeric scores (0.0 to 1.0), apply per-dimension weights, and calculate weighted overall scores. Set passing thresholds based on use-case requirements, typically 0.7 for general use and 0.9 for high-stakes applications. Store individual dimension scores alongside the aggregate because the breakdown drives targeted improvement.

构建多维 Rubric

定义覆盖关键维度的 rubric，并为从优秀到失败的各个等级提供清晰的描述性定义。纳入以下核心维度，并根据具体用例调整权重：

事实准确性：陈述与 ground truth 一致（知识类任务应赋予较高权重）
完整性：输出覆盖了要求的所有方面（研究类任务应赋予较高权重）
引用准确性：引用与所声称的来源匹配（信任敏感场景应赋予一定权重）
来源质量：使用适当的一手来源（权威性输出应赋予较高权重）
工具效率：以合理次数使用正确的工具（成本敏感型系统应赋予较高权重）

将 Rubric 转换为数值分数

将各维度的评估结果映射为数值分数（0.0 到 1.0），应用各维度权重，并计算加权总体分数。根据用例要求设置通过阈值，通常通用场景为 0.7，高风险场景为 0.9。由于分项结果会驱动定向改进，因此在存储汇总分数的同时，也应存储各维度的单独分数。

Evaluation Methodologies

评估方法

Use LLM-as-Judge for Scale

Build LLM-based evaluation prompts that include: clear task description, the agent output under test, ground truth when available, an evaluation scale with explicit level descriptions, and a request for structured judgment with reasoning. LLM judges provide consistent, scalable evaluation across large test sets. Use a different model family than the agent being evaluated to avoid self-enhancement bias.

Supplement with Human Evaluation

Route edge cases, unusual queries, and a random sample of production traffic to human reviewers because humans notice hallucinated answers, system failures, and subtle biases that automated evaluation misses. Track patterns across human reviews to identify systematic issues and feed findings back into automated evaluation criteria.

Apply End-State Evaluation for Stateful Agents

For agents that mutate persistent state (files, databases, configurations), evaluate whether the final state matches expectations rather than how the agent got there. Define expected end-state assertions and verify them programmatically after each test run.

使用 LLM-as-Judge 实现规模化评估

构建基于 LLM 的评估 prompts，其中应包括：清晰的任务描述、待测试的 Agent 输出、可用时的 ground truth、带明确等级说明的评估量表，以及要求返回包含推理过程的结构化判断。LLM judges 可在大规模测试集上提供一致且可扩展的评估。应使用与被评估 Agent 不同的模型家族，以避免自我强化偏差。

辅以人工评估

将边缘案例、异常查询和随机抽取的生产流量交给人工审核，因为人工能够发现自动评估遗漏的幻觉答案、系统故障和细微偏差。跟踪人工审核中的模式，以识别系统性问题，并将发现反馈到自动评估标准中。

对有状态 Agent 应用终态评估

对于会修改持久状态（文件、数据库、配置）的 Agent，应评估最终状态是否符合预期，而不是关注 Agent 如何到达该状态。定义预期的终态断言，并在每次测试运行后通过程序进行验证。

Test Set Design

测试集设计

Select Representative Samples

Start with small samples (20-30 cases) during early development when changes have dramatic impacts and low-hanging fruit is abundant. Scale to 50+ cases for reliable signal as the system matures. Sample from real usage patterns, add known edge cases, and ensure coverage across complexity levels.

Stratify by Complexity

Structure test sets across complexity levels to prevent easy examples from inflating scores:

Simple: single tool call, factual lookup
Medium: multiple tool calls, comparison logic
Complex: many tool calls, significant ambiguity
Very complex: extended interaction, deep reasoning, synthesis

Report scores per stratum alongside overall scores to reveal where the agent actually struggles.

选择代表性样本

在早期开发阶段，当变更影响显著且容易获得改进机会时，先使用小规模样本（20-30 个案例）。随着系统成熟，扩展到 50 个以上案例，以获得可靠信号。样本应来自真实使用模式，加入已知边缘案例，并确保覆盖各个复杂度级别。

按复杂度分层

构建跨复杂度层级的测试集，防止简单示例抬高分数：

简单：单次工具调用，事实查询
中等：多次工具调用，比较逻辑
复杂：大量工具调用，存在显著歧义
非常复杂：长时间交互，深度推理，综合分析

除总体分数外，还要按层报告分数，以揭示 Agent 实际在哪些方面遇到困难。

Context Engineering Evaluation

Context Engineering 评估

Validate Context Strategies Systematically

Run agents with different context strategies on the same test set and compare quality scores, token usage, and efficiency metrics. This isolates the effect of context engineering from other variables and prevents anecdote-driven decisions.

Run Degradation Tests

Test how context degradation affects performance by running agents at different context sizes. Identify performance cliffs where context becomes problematic and establish safe operating limits. Feed these limits back into context management strategies.

系统验证 Context 策略

在相同测试集上使用不同的 context 策略运行 Agent，并比较质量分数、token 使用量和效率指标。这样可以隔离 context engineering 的作用，避免其他变量干扰，并防止仅凭个案经验做决策。

运行退化测试

通过在不同 context 大小下运行 Agent，测试 context 退化对性能的影响。找出 context 开始造成问题的性能断崖，并确定安全运行限制。将这些限制反馈到 context 管理策略中。

Continuous Evaluation

持续评估

Build Automated Evaluation Pipelines

Integrate evaluation into the development workflow so evaluations run automatically on agent changes. Track results over time, compare versions, and block deployments that regress on key metrics.

Monitor Production Quality

Sample production interactions and evaluate them continuously. Set alerts for quality drops below warning (0.85 pass rate) and critical (0.70 pass rate) thresholds. Maintain dashboards showing trend analysis over time windows to detect gradual degradation.

构建自动化评估流水线

将评估集成到开发工作流中，使 Agent 发生变更时评估自动运行。持续跟踪结果、比较版本，并阻止在关键指标上出现回归的部署。

监控生产质量

持续抽样并评估生产交互。为质量低于警告阈值（0.85 通过率）和严重阈值（0.70 通过率）的情况设置警报。维护仪表板，按不同时间窗口展示趋势分析，以发现逐渐发生的退化。

Practical Guidance

实践指导

Building Evaluation Frameworks

构建评估框架

Follow this sequence to build an evaluation framework, because skipping early steps leads to measurements that do not reflect real quality:

Define quality dimensions relevant to the use case before writing any evaluation code, because dimensions chosen later tend to reflect what is easy to measure rather than what matters.
Create rubrics with clear, descriptive level definitions so evaluators (human or LLM) produce consistent scores.
Build test sets from real usage patterns and edge cases, stratified by complexity, with at least 50 cases for reliable signal.
Implement automated evaluation pipelines that run on every significant change.
Establish baseline metrics before making changes so improvements can be measured against a known reference.
Run evaluations on all significant changes and compare against the baseline.
Track metrics over time for trend analysis because gradual degradation is harder to notice than sudden drops.
Supplement automated evaluation with human review on a regular cadence.
Separate deterministic validation failures from quality judgments so invalid artifacts cannot be laundered by a favorable LLM score.

请按以下顺序构建评估框架，因为跳过早期步骤会导致测量结果无法反映真实质量：

在编写任何评估代码之前，定义与用例相关的质量维度，因为之后才选择的维度往往会反映容易衡量的内容，而不是真正重要的内容。
创建带有清晰、可描述等级定义的 rubric，使评估者（人工或 LLM）生成一致的分数。
根据真实使用模式和边缘案例构建测试集，按复杂度分层，并至少包含 50 个案例，以获得可靠信号。
实现自动化评估流水线，在每次重大变更时运行。
在进行变更前建立基线指标，以便针对已知参考衡量改进效果。
对所有重大变更运行评估，并与基线比较。
随时间跟踪指标以进行趋势分析，因为渐进式退化比突然下降更难发现。
定期以人工审核补充自动化评估。
将 deterministic validation 失败与质量判断分离开来，避免无效产物通过有利的 LLM 分数被“洗白”。

Avoiding Evaluation Pitfalls

避免评估陷阱

Guard against these common failures that undermine evaluation reliability:

Overfitting to specific paths: Evaluate outcomes, not specific steps, because agents find novel valid paths.
Ignoring edge cases: Include diverse test scenarios covering the full complexity spectrum.
Single-metric obsession: Use multi-dimensional rubrics because a single score hides dimension-specific failures.
Neglecting context effects: Test with realistic context sizes and histories rather than clean-room conditions.
Skipping human evaluation: Automated evaluation misses subtle issues that humans catch reliably.

防范以下常见问题，以免削弱评估可靠性：

对特定路径过拟合：评估结果，而不是具体步骤，因为 Agent 会找到新颖的有效路径。
忽视边缘案例：纳入覆盖完整复杂度范围的多样化测试场景。
执着于单一指标：使用多维 rubric，因为单一分数会掩盖特定维度上的失败。
忽视 context 效应：使用符合实际的 context 大小和历史记录进行测试，而不是在洁净室条件下测试。
跳过人工评估：自动评估会遗漏人工能够可靠发现的细微问题。

Examples

示例

Example 1: Simple Evaluation

python

def evaluate_agent_response(response, expected):
    rubric = load_rubric()
    scores = {}
    for dimension, config in rubric.items():
        scores[dimension] = assess_dimension(response, expected, dimension)
    overall = weighted_average(scores, config["weights"])
    return {"passed": overall >= 0.7, "scores": scores}

Example 2: Test Set Structure

Test sets should span multiple complexity levels to ensure comprehensive evaluation:

python

test_set = [
    {
        "name": "simple_lookup",
        "input": "What is the capital of France?",
        "expected": {"type": "fact", "answer": "Paris"},
        "complexity": "simple",
        "description": "Single tool call, factual lookup"
    },
    {
        "name": "medium_query",
        "input": "Compare the revenue of Apple and Microsoft last quarter",
        "complexity": "medium",
        "description": "Multiple tool calls, comparison logic"
    },
    {
        "name": "multi_step_reasoning",
        "input": "Analyze sales data from Q1-Q4 and create a summary report with trends",
        "complexity": "complex",
        "description": "Many tool calls, aggregation, analysis"
    },
    {
        "name": "research_synthesis",
        "input": "Research emerging AI technologies, evaluate their potential impact, and recommend adoption strategy",
        "complexity": "very_complex",
        "description": "Extended interaction, deep reasoning, synthesis"
    }
]

Example 3: Deterministic gate before model judgment

python

def evaluate_pr_candidate(candidate):
    structure = run_validate_repo(candidate)
    if not structure.ok:
        return {"passed": False, "reason": "deterministic validation failed", "details": structure.errors}

    quality = run_rubric_eval(candidate)
    return {"passed": quality.overall >= 0.8, "scores": quality.dimensions}

Example 4: Quality gate dimensions

yaml

gate:
  deterministic:
    - schema_valid
    - required_files_present
    - no_duplicate_ids
  quality:
    factual_accuracy: min 0.85
    completeness: min 0.80
    source_traceability: min 0.90

示例 1：简单评估

python

def evaluate_agent_response(response, expected):
    rubric = load_rubric()
    scores = {}
    for dimension, config in rubric.items():
        scores[dimension] = assess_dimension(response, expected, dimension)
    overall = weighted_average(scores, config["weights"])
    return {"passed": overall >= 0.7, "scores": scores}

示例 2：测试集结构

测试集应跨越多个复杂度级别，以确保评估全面：

python

test_set = [
    {
        "name": "simple_lookup",
        "input": "What is the capital of France?",
        "expected": {"type": "fact", "answer": "Paris"},
        "complexity": "simple",
        "description": "Single tool call, factual lookup"
    },
    {
        "name": "medium_query",
        "input": "Compare the revenue of Apple and Microsoft last quarter",
        "complexity": "medium",
        "description": "Multiple tool calls, comparison logic"
    },
    {
        "name": "multi_step_reasoning",
        "input": "Analyze sales data from Q1-Q4 and create a summary report with trends",
        "complexity": "complex",
        "description": "Many tool calls, aggregation, analysis"
    },
    {
        "name": "research_synthesis",
        "input": "Research emerging AI technologies, evaluate their potential impact, and recommend adoption strategy",
        "complexity": "very_complex",
        "description": "Extended interaction, deep reasoning, synthesis"
    }
]

示例 3：模型判断前的确定性门禁

python

def evaluate_pr_candidate(candidate):
    structure = run_validate_repo(candidate)
    if not structure.ok:
        return {"passed": False, "reason": "deterministic validation failed", "details": structure.errors}

    quality = run_rubric_eval(candidate)
    return {"passed": quality.overall >= 0.8, "scores": quality.dimensions}

示例 4：质量门禁维度

yaml

gate:
  deterministic:
    - schema_valid
    - required_files_present
    - no_duplicate_ids
  quality:
    factual_accuracy: min 0.85
    completeness: min 0.80
    source_traceability: min 0.90

Guidelines

指南

Use multi-dimensional rubrics, not single metrics
Evaluate outcomes, not specific execution paths
Cover complexity levels from simple to complex
Test with realistic context sizes and histories
Run evaluations continuously, not just before release
Supplement LLM evaluation with human review
Track metrics over time for trend detection
Set clear pass/fail thresholds based on use case

使用多维 rubric，而不是单一指标
评估结果，而不是特定执行路径
覆盖从简单到复杂的各个复杂度级别
使用符合实际的 context 大小和历史记录进行测试
持续运行评估，而不是只在发布前运行
以人工审核补充 LLM 评估
随时间跟踪指标以检测趋势
根据用例设置清晰的通过/失败阈值

Gotchas

常见陷阱

Overfitting evals to specific code paths: Tests pass but the agent fails on slight input variations. Write eval criteria against outcomes and semantics, not surface patterns, and rotate test inputs periodically.
LLM-judge self-enhancement bias: Models rate their own outputs higher than independent judges do. Use a different model family as the evaluation judge than the model being evaluated.
Test set contamination: Eval examples leak into training data or prompt templates, inflating scores. Keep eval sets versioned and separate from any data used in prompts or fine-tuning.
Metric gaming: Optimizing for the metric rather than actual quality produces agents that score well but disappoint users. Cross-validate automated metrics against human judgments regularly.
Single-dimension scoring: One aggregate number hides critical failures in specific dimensions. Always report per-dimension scores alongside the overall score, and fail the eval if any single dimension falls below its minimum threshold.
Eval set too small: Fewer than 50 examples produces unreliable signal with high variance between runs. Scale the eval set to at least 50 cases and report confidence intervals.
Not stratifying by difficulty: Easy examples inflate overall scores, masking failures on hard cases. Report scores per complexity stratum and weight the overall score to prevent easy-case dominance.
Treating eval as one-time: Evaluation must be continuous, not a launch gate. Agent quality drifts as models update, tools change, and usage patterns evolve. Run evals on every change and on a regular production cadence.

评估对特定代码路径过拟合：测试通过，但输入略有变化时 Agent 就会失败。评估标准应针对结果和语义，而不是表面模式，并定期轮换测试输入。
LLM-judge 自我强化偏差：模型对自身输出的评分往往高于独立 judge。使用与被评估模型不同的模型家族作为评估 judge。
测试集污染：评估示例泄漏到训练数据或 prompt 模板中，从而抬高分数。应对评估集进行版本管理，并将其与用于 prompt 或微调的任何数据分离。
指标博弈：针对指标而非实际质量进行优化，会产生分数很高却让用户失望的 Agent。应定期使用人工判断交叉验证自动指标。
单维度评分：一个汇总数字会掩盖特定维度中的严重失败。除总体分数外，始终报告各维度分数；如果任一维度低于其最低阈值，则评估应失败。
评估集过小：少于 50 个示例会导致信号不可靠，且不同运行之间方差很高。将评估集扩展到至少 50 个案例，并报告置信区间。
未按难度分层：简单示例会抬高总体分数，掩盖困难案例中的失败。按复杂度层报告分数，并对总体分数加权，以防止简单案例占据主导。
将评估视为一次性任务：评估必须持续进行，而不是仅作为发布门禁。随着模型更新、工具变化和使用模式演变，Agent 质量会发生漂移。每次变更时都运行评估，并按固定生产节奏运行。

Integration

集成

This skill owns outcome measurement and quality gates. Adjacent skills own specialized evaluator design and control-loop governance:

```
advanced-evaluation
```
: LLM-as-judge prompt design, pairwise comparison, calibration, and bias mitigation.
```
harness-engineering
```
: locked evaluators, editable surfaces, rollback, and human approval boundaries.
```
context-degradation
```
: detecting and measuring degradation patterns.
```
context-optimization
```
: measuring token, cost, latency, and quality effects of optimizations.
```
multi-agent-patterns
```
: evaluating coordination quality and parallelization trade-offs.
```
tool-design
```
: evaluating tool selection and recovery effectiveness.
```
memory-systems
```
: evaluating memory retrieval and retention quality.

此 skill 负责结果衡量和质量门禁。相邻 skill 负责专业评估器设计和控制循环治理：

```
advanced-evaluation
```
：LLM-as-judge prompt 设计、成对比较、校准和偏差缓解。
```
harness-engineering
```
：锁定的评估器、可编辑界面、rollback 和人工审批边界。
```
context-degradation
```
：检测和衡量退化模式。
```
context-optimization
```
：衡量优化对 token、成本、延迟和质量的影响。
```
multi-agent-patterns
```
：评估协调质量和并行化权衡。
```
tool-design
```
：评估工具选择和恢复机制的有效性。
```
memory-systems
```
：评估记忆检索和保留质量。

References

参考资料

Internal reference:

Metrics Reference - Read when: designing specific evaluation metrics, choosing scoring scales, or implementing weighted rubric calculations

Internal skills:

All other skills connect to evaluation for quality measurement

External resources:

LLM evaluation benchmarks - Read when: selecting or building benchmark suites for agent comparison
Agent evaluation research papers - Read when: adopting new evaluation methodologies or validating current approach
Production monitoring practices - Read when: setting up alerting, dashboards, or sampling strategies for live systems

内部参考：

指标参考 - 阅读时机：设计具体评估指标、选择评分量表或实现加权 rubric 计算

内部 skill：

所有其他 skill 都通过评估与质量衡量相连接

外部资源：

LLM 评估基准 - 阅读时机：选择或构建用于 Agent 比较的基准测试套件
Agent 评估研究论文 - 阅读时机：采用新的评估方法或验证当前方法
生产监控实践 - 阅读时机：为线上系统设置警报、仪表板或采样策略

Skill Metadata

Skill 元数据

Created: 2025-12-20 Last Updated: 2026-05-15 Author: Agent Skills for Context Engineering Contributors Version: 1.2.0

创建时间：2025-12-20 最后更新时间：2026-05-15 作者：Agent Skills for Context Engineering Contributors 版本：1.2.0