eval-relevance

Original：🇺🇸 English

Translated

Score assistant responses for relevance on a strict 1-5 scale, then return strict JSON only with score, rationale, and improvement suggestions. Use when the user asks to evaluate relevance, grade relevance, or critique topical alignment.

1installs

Sourcewhitespectre/ai-assistant-evals

Added on2026-02-19

NPX Install

npx skill4agent add whitespectre/ai-assistant-evals eval-relevance

SKILL.md Content

View Translation Comparison →

Eval Relevance

Use this skill to evaluate how relevant an assistant response is to the user’s request.

Inputs

Require:

The assistant response text to evaluate.
(Optional) The user’s original request for comparison.

Internal Rubric (1–5)

5 = Directly addresses the user’s request, stays fully on-topic, and prioritizes what the user actually asked
4 = Mostly relevant, minor digressions or small omissions
3 = Partially relevant, addresses the general topic but misses key parts of the request
2 = Weak relevance, significant digressions or failure to address the core request
1 = Not relevant, does not address the user’s request or answers a different question entirely

Workflow

Compare the assistant response to the user’s request (if provided).
Score relevance on a 1-5 integer scale using the rubric only.
Write concise rationale tied directly to rubric criteria.
Produce actionable suggestions that improve relevance.

Output Contract

Return JSON only. Do not include markdown, backticks, prose, or extra keys.

Use exactly this schema:

{ "dimension": "relevance", "score": 1, "rationale": "...", "improvement_suggestions": [ "..." ] }

Hard Rules

```
dimension
```
must always equal
```
"relevance"
```
.
```
score
```
must be an integer from 1 to 5.
```
rationale
```
must be concise (max 3 sentences).
Do not include step-by-step reasoning.
```
improvement_suggestions
```
must be a non-empty array of concrete edits.
Never output text outside the JSON object.