agent-self-evaluation

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Agent Self-Evaluation

Agent自我评估

After completing a complex task, the agent pauses to rate its own output against a structured 5-axis rubric. This is NOT a pass/fail gate — it's a deliberate reflection step that catches omissions, flags overconfidence, and surface areas for improvement before the user has to.

完成复杂任务后，Agent会暂停下来，对照结构化的5维度评分标准对自身输出进行评估。这并非合格/不合格的关卡——而是一个刻意的反思步骤，能在用户发现之前找出遗漏问题、标记过度自信的情况，并明确需要改进的方向。

When to Activate

激活时机

After writing code that spans 3+ files or 50+ lines
After completing a multi-step workflow (implement → test → review)
After a debugging session that involved 3+ attempts
After producing a design document, architecture decision, or written analysis
When the user asks "how good was that?" or "rate yourself"
At the end of any session Stop hook (if configured — see
```
references/hook-integration.md
```
)

编写跨3个及以上文件或50行及以上代码后
完成多步骤工作流（实现→测试→评审）后
涉及3次及以上尝试的调试会话结束后
生成设计文档、架构决策或书面分析后
当用户询问“这个成果怎么样？”或“给你自己打分”时
任何会话Stop钩子结束时（若已配置——详见
```
references/hook-integration.md
```
）

Core Concepts

核心概念

The 5 Evaluation Axes

5个评估维度

Axis	Question	What it catches
Accuracy	Are the facts, claims, and outputs correct?	Hallucinations, wrong API names, incorrect syntax, false statements
Completeness	Did it cover everything the user asked for?	Missed edge cases, unhandled error paths, forgotten requirements, skipped subtasks
Clarity	Is the explanation understandable and well-structured?	Confusing explanations, jargon without definition, missing context, rambling
Actionability	Can the user act on the output immediately?	Vague suggestions, missing steps, "you should X" without showing how, no verification path
Conciseness	Did it use the minimum words/tokens needed?	Redundancy, over-explanation, repeating the user's question verbatim, filler content

维度	问题	检测内容
Accuracy（准确性）	事实、声明和输出是否正确？	幻觉内容、错误的API名称、不正确的语法、虚假陈述
Completeness（完整性）	是否覆盖了用户要求的所有内容？	遗漏的边缘情况、未处理的错误路径、被遗忘的需求、跳过的子任务
Clarity（清晰度）	解释是否易懂且结构清晰？	令人困惑的解释、未定义的行话、缺失的上下文、冗长的表述
Actionability（可操作性）	用户能否立即根据输出采取行动？	模糊的建议、缺失的步骤、只说“你应该做X”却不展示方法、无验证路径
Conciseness（简洁性）	是否使用了最少的文字/令牌？	冗余内容、过度解释、逐字重复用户问题、填充内容

Scoring Scale

评分尺度

5 — Exceptional: no reasonable improvement possible
4 — Good: minor nits only, no substantive gaps
3 — Adequate: meets the request but has a notable weakness on at least one axis
2 — Weak: has a clear gap that affects usability or correctness
1 — Poor: fundamentally misses the request or contains significant errors

5 — 优秀：无合理改进空间
4 — 良好：仅存在小瑕疵，无实质性漏洞
3 — 合格：满足需求但至少在一个维度上存在明显弱点
2 — 薄弱：存在影响可用性或正确性的明确漏洞
1 — 较差：完全未满足需求或包含重大错误

The Evidence Rule

证据规则

Every score below 5 MUST cite specific evidence. A score of 3 cannot just say "could be better" — it must say exactly what is missing or wrong. The mantra: "Show the gap, don't just name it."

所有低于5分的评分必须引用具体证据。评3分时不能只说“可以更好”——必须明确指出缺失或错误的具体内容。准则：“指出漏洞，而非仅命名问题。”

Workflow

工作流程

Step 1: Collect the Raw Material

步骤1：收集原始素材

Gather what you'll evaluate:

- The original user request (read back from conversation)
- Your final response/output (the deliverable)
- Any tool outputs that verify correctness (test results, exit codes, lint output)
- Any user feedback received during the task (corrections, "try again", "that's not right")

整理需要评估的内容：

- 原始用户请求（从对话中回溯）
- 最终响应/输出（交付成果）
- 任何验证正确性的工具输出（测试结果、退出码、lint输出）
- 任务过程中收到的任何用户反馈（修正意见、“重试”、“不对”）

Step 2: Score Each Axis Independently

步骤2：独立为每个维度评分

Work through the 5 axes one at a time. For each:

Read the axis question
Find evidence (or lack of evidence) in the output
Assign a score 1-5
If score < 5, write a one-sentence improvement note citing the gap

Do NOT average the scores in your head first and then work backwards. Score each axis fresh.

逐个处理5个维度。针对每个维度：

阅读维度对应的问题
在输出中寻找证据（或缺乏证据的情况）
给出1-5分的评分
若评分<5，撰写一句引用漏洞的改进说明

请勿先在脑海中计算平均分再倒推评分。需为每个维度重新评分。

Step 3: Produce the Evaluation Report

步骤3：生成评估报告

Use the template from

templates/evaluation-report.md

. The report must include:

- One-line summary
- 5-axis scorecard (score + evidence per axis)
- Overall score (simple average, rounded to 1 decimal)
- 1-3 specific improvements ranked by impact
- Self-check: "Would the user agree with this assessment?"

使用

templates/evaluation-report.md

中的模板。报告必须包含：

- 一句话总结
- 5维度评分卡（每个维度含评分+证据）
- 总体评分（简单平均值，保留1位小数）
- 1-3条按影响优先级排序的具体改进建议
- 自我检查：“用户是否会同意此评估？”

Step 4: Apply the Improvement

步骤4：落实改进措施

If any axis scored 3 or below:

State what you would do differently
If the gap is fixable in < 30 seconds (missing link, unclear phrasing), fix it now
If the gap requires rework, flag it explicitly: "This axis scored [reason] because [evidence]. Re-running with [specific fix] would likely raise it to [score]."

若任何维度评分≤3：

说明你会做出哪些不同的处理
若漏洞可在30秒内修复（缺失链接、表述模糊），立即修复
若漏洞需要返工，明确标记：“此维度评分为[分数]，原因是[证据]。通过[具体修复方案]重新执行，评分可能提升至[目标分数]。”

Code Examples

代码示例

Example: Good Evaluation (Score 4+)

示例：良好评估（评分4+）

Task: Add retry logic to HTTP client

Scorecard:
  Accuracy:    5 — All API calls correct. Verified: retries use
                  exponential backoff. No hallucinated methods.
  Completeness: 4 — Covered happy path + 3 error cases. Missing:
                  timeout handling for hung connections.
  Clarity:      5 — Code comments explain backoff formula.
                  PR description links to incident that motivated this.
  Actionability:5 — Single merge. No follow-up tasks. Tests pass.
  Conciseness:  4 — 47 lines total. The retry loop could be extracted
                  into a helper to drop ~8 lines.

Overall: 4.6 — One gap (timeout handling). Fix before merging.

Task: Add retry logic to HTTP client

Scorecard:
  Accuracy:    5 — All API calls correct. Verified: retries use
                  exponential backoff. No hallucinated methods.
  Completeness: 4 — Covered happy path + 3 error cases. Missing:
                  timeout handling for hung connections.
  Clarity:      5 — Code comments explain backoff formula.
                  PR description links to incident that motivated this.
  Actionability:5 — Single merge. No follow-up tasks. Tests pass.
  Conciseness:  4 — 47 lines total. The retry loop could be extracted
                  into a helper to drop ~8 lines.

Overall: 4.6 — One gap (timeout handling). Fix before merging.

Example: Weak Evaluation (Score 2-3)

示例：薄弱评估（评分2-3）

Task: Add retry logic to HTTP client

Scorecard:
  Accuracy:    2 — Used urllib3 which doesn't match our
                  httpx-based codebase. Wrong library.
  Completeness: 3 — Works for GET. POST/PUT not handled (user
                  said "all HTTP requests").
  Clarity:      4 — Code is readable. Good variable names.
  Actionability:2 — "Add tests" mentioned but no test file created.
                  User has to write tests before merging.
  Conciseness:  3 — 120 lines. The retry config is duplicated in
                  3 places instead of one shared RetryConfig object.

Overall: 2.8 — Wrong library used. Needs httpx rewrite.
  Fix accuracy first (switch to httpx), then extend to all
  HTTP methods, then consolidate config.

Task: Add retry logic to HTTP client

Scorecard:
  Accuracy:    2 — Used urllib3 which doesn't match our
                  httpx-based codebase. Wrong library.
  Completeness: 3 — Works for GET. POST/PUT not handled (user
                  said "all HTTP requests").
  Clarity:      4 — Code is readable. Good variable names.
  Actionability:2 — "Add tests" mentioned but no test file created.
                  User has to write tests before merging.
  Conciseness:  3 — 120 lines. The retry config is duplicated in
                  3 places instead of one shared RetryConfig object.

Overall: 2.8 — Wrong library used. Needs httpx rewrite.
  Fix accuracy first (switch to httpx), then extend to all
  HTTP methods, then consolidate config.

Anti-Patterns

反模式

"Everything is a 5"

“所有维度都是5分”

FAIL: Accuracy:    5 — All good.
   Completeness: 5 — Everything covered.
   Clarity:      5 — Clear.

No evidence cited. This is self-congratulation, not evaluation. A real 5 requires proving there's nothing to improve.

FAIL: Accuracy:    5 — All good.
   Completeness: 5 — Everything covered.
   Clarity:      5 — Clear.

无证据引用。这是自我吹捧，而非评估。真正的5分需要证明不存在任何改进空间。

Over-penalizing for scope creep

因范围扩张过度扣分

FAIL: Completeness: 2 — Didn't handle WebSocket connections or
   gRPC streaming (user didn't ask for these)

Only evaluate against what the user actually requested, not what you could have additionally built.

FAIL: Completeness: 2 — Didn't handle WebSocket connections or
   gRPC streaming (user didn't ask for these)

仅针对用户实际请求的内容进行评估，而非额外可拓展的功能。

Using the evaluation to re-litigate

利用评估重新争论

FAIL: "As I said earlier, this approach is wrong. Score: 1"

The evaluation is about the delivered output, not about re-arguing design decisions that were already made. If the approach was wrong, that should have been caught before delivery.

FAIL: "As I said earlier, this approach is wrong. Score: 1"

评估针对的是交付的输出，而非重新争论已做出的设计决策。若方法有误，应在交付前就发现。

Mixing personal preference with objective gaps

将个人偏好与客观漏洞混淆

FAIL: "Score: 3. I don't like Python decorators."

"Don't like" is not evidence. Cite a concrete readability, testability, or correctness concern, or leave the score at 4+.

FAIL: "Score: 3. I don't like Python decorators."

“不喜欢”并非证据。需引用具体的可读性、可测试性或正确性问题，否则评分应≥4。

Best Practices

最佳实践

Evaluate the output, not the process. The user cares about what you delivered, not how many iterations you took.
One improvement per weak axis. Don't list 5 things for one axis — pick the highest-impact gap.
Tie improvements to user impact. "Missing error handling means the user's API call will crash silently" beats "add error handling."
Be specific about what 'fixed' looks like. "Re-run with httpx transport configured for retries" beats "fix the library issue."
Use tool outputs as evidence. If tests passed, cite them. If lint is clean, cite it. Don't guess — grep for the proof.
If you can't find any gaps, try harder. A perfect score across all 5 axes is rare. Ask: "If I were the user, what would annoy me about this output?"

评估输出，而非过程。 用户关心的是交付成果，而非你经历了多少次迭代。
每个薄弱维度对应一条改进建议。 不要为一个维度列出5个问题——选择影响最大的漏洞。
将改进与用户影响关联。 “缺失错误处理会导致用户的API调用静默崩溃”比“添加错误处理”更有说服力。
明确说明‘修复后’的状态。 “使用配置了重试的httpx传输重新执行”比“修复库问题”更具体。
使用工具输出作为证据。 若测试通过，引用测试结果；若lint检查通过，引用检查结果。不要猜测——寻找确凿证据。
若未找到任何漏洞，再深入挖掘。 所有维度都得满分的情况极为罕见。试问：“如果我是用户，这个输出有什么会让我不满的地方？”