eval-model

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Evaluate a model's accuracy on MAX

评估MAX上模型的准确率

A model that serves cleanly can still answer benchmark questions wrong. This skill measures how accurately a text model behind MAX answers standard datasets — GSM8K, MMLU, HellaSwag, ARC, AIME, GPQA, TruthfulQA, WinoGrande, and BABILong. It checks endpoint compatibility before each run, separates serving failures from wrong answers, and writes reproducible per-task scores you can compare against a model card.

Use this skill when you're benchmarking a served model, comparing it with model-card or reference scores, verifying that a newly imported MAX model answers correctly, or running repeatable dataset evaluations against a MAX OpenAI-compatible endpoint.

Do not use this skill when the model doesn't serve yet. Bring the model up first (

import-model

); if it serves but generates wrong output, chase the divergence with

debug-model

before benchmarking.

一个能正常提供服务的模型仍可能在基准测试问题上给出错误答案。本Skill可衡量由MAX后端提供服务的文本模型在标准数据集上的回答准确率——这些数据集包括GSM8K、MMLU、HellaSwag、ARC、AIME、GPQA、TruthfulQA、WinoGrande和BABILong。它会在每次运行前检查端点兼容性，区分服务故障与错误答案，并生成可复现的单任务分数，供你与模型卡片进行对比。

适用场景：对已部署的模型进行基准测试、将其与模型卡片或参考分数对比、验证新导入的MAX模型回答是否正确，或是针对MAX兼容OpenAI的端点运行可重复的数据集评估。

不适用场景：模型尚未完成部署。请先启动模型（执行

import-model

）；若模型已部署但生成错误输出，请先使用

debug-model

排查问题，再进行基准测试。

References

参考文档

File	Read when
references/examples.md	You want more command variants — remote endpoints, artifact paths, long context
references/fast-fail-thresholds.md	Before changing seed floors or adding a seed task
references/heartbeat-protocol.md	Building a program that consumes the progress stream

Read the reference for what you're doing, not all of them upfront.

文件	阅读时机
references/examples.md	需要更多命令变体——如远程端点、工件路径、长上下文相关命令时
references/fast-fail-thresholds.md	修改种子阈值或添加种子任务之前
references/heartbeat-protocol.md	构建消费进度流的程序时

按需阅读对应参考文档，无需提前通读全部内容。

Set up the evaluator

搭建评估器环境

Resolve the absolute path to the directory containing this

SKILL.md

and save it as

SKILL_DIR

. Every command below depends on it, so set it from the skill's real location — not the user's working directory and not the literal placeholder below. If you were handed the skill path, export that directly; otherwise resolve it from the path to this file:

bash

export SKILL_DIR="$(cd "$(dirname /path/to/eval-model/SKILL.md)" && pwd)"

Confirm it points at the manifest before continuing, so a wrong value fails here rather than midway through a run:

bash

test -f "$SKILL_DIR/pixi.toml" && echo "SKILL_DIR ok: $SKILL_DIR"

Install the evaluator environment from its Pixi manifest:

bash

pixi install --manifest-path "$SKILL_DIR/pixi.toml"

This environment holds the evaluator's own dependencies (lm-eval, datasets, transformers).

Some datasets require internet access, Hugging Face authentication, or license acceptance. Set

HF_TOKEN

before the run when the dataset requires it.

获取包含本

SKILL.md

的目录绝对路径并保存为

SKILL_DIR

。以下所有命令均依赖该路径，请确保从Skill的实际位置设置，而非用户工作目录或下方的占位符。若已获取Skill路径，直接导出即可；否则通过本文件路径解析：

bash

export SKILL_DIR="$(cd "$(dirname /path/to/eval-model/SKILL.md)" && pwd)"

继续前请确认该路径指向配置清单，避免中途因路径错误导致失败：

bash

test -f "$SKILL_DIR/pixi.toml" && echo "SKILL_DIR ok: $SKILL_DIR"

通过Pixi配置清单安装评估器环境：

bash

pixi install --manifest-path "$SKILL_DIR/pixi.toml"

该环境包含评估器自身的依赖（lm-eval、datasets、transformers）。

部分数据集需要联网、Hugging Face认证或许可证授权。若数据集有此要求，请在运行前设置

HF_TOKEN

。

Serve the model

部署模型

The tasks fall into two groups that serve the model differently — decide which you're running, or serve so you can run both:

Generation tasks (
```
gsm8k
```
,
```
aime
```
) score a generated answer and run against a normally served model.
Multiple-choice tasks (
```
mmlu
```
,
```
hellaswag
```
,
```
arc_easy
```
,
```
arc_challenge
```
,
```
winogrande
```
,
```
truthfulqa
```
,
```
gpqa
```
) score by prompt log probabilities, which MAX only returns when the server starts with
```
--enable-echo
```
.

Serve from an environment where MAX is installed.

For generation tasks, start the model normally:

bash

max serve --model meta-llama/Llama-3.1-8B-Instruct

For multiple-choice tasks, add

--enable-echo

bash

max serve \
  --model meta-llama/Llama-3.1-8B-Instruct \
  --enable-echo

You can't toggle echo on a live server. If one is already running without

--enable-echo

, stop and restart it before evaluating a multiple-choice task — there's no way to add the capability to a running process. The evaluator's preflight is the authoritative check: it verifies prompt log probabilities before downloading datasets and reports a configuration error if they're unavailable.

If a model's

generation_config.json

sets a non-default

repetition_penalty

(the Qwen2.5 family ships

1.05

, for example), MAX auto-enables its penalty sampling path, which is shape-incompatible with echo and crashes the serving worker at startup with a

repetition_penalty

tensor-shape error. Models that leave

repetition_penalty

unset (such as Llama-3.1 and Qwen3) don't trigger this path. If you hit the crash, evaluate a model whose generation config leaves

repetition_penalty

unset, or restrict the run to generation tasks (

gsm8k

aime

任务分为两类，对应不同的模型部署方式——请确定要运行的任务类型，或按支持两类任务的方式部署模型：

生成类任务（
```
gsm8k
```
、
```
aime
```
）：对生成的答案打分，可在正常部署的模型上运行。
多选类任务（
```
mmlu
```
、
```
hellaswag
```
、
```
arc_easy
```
、
```
arc_challenge
```
、
```
winogrande
```
、
```
truthfulqa
```
、
```
gpqa
```
）：通过提示词对数概率打分，MAX仅在服务器启动时添加
```
--enable-echo
```
参数才会返回该数据。

请在已安装MAX的环境中部署模型。

针对生成类任务，正常启动模型：

bash

max serve --model meta-llama/Llama-3.1-8B-Instruct

针对多选类任务，添加

--enable-echo

参数：

bash

max serve \
  --model meta-llama/Llama-3.1-8B-Instruct \
  --enable-echo

无法在运行中的服务器上切换echo功能。若服务器已启动且未添加

--enable-echo

，请先停止再重启，才能评估多选类任务——运行中的进程无法添加该功能。评估器的预检查会进行权威验证：它会在下载数据集前验证提示词对数概率是否可用，若不可用则报告配置错误。

若模型的

generation_config.json

设置了非默认的

repetition_penalty

（例如Qwen2.5系列默认值为

1.05

），MAX会自动启用惩罚采样路径，该路径与echo功能不兼容，会导致服务进程启动时因

repetition_penalty

张量形状错误崩溃。未设置

repetition_penalty

的模型（如Llama-3.1和Qwen3）不会触发此问题。若遇到崩溃，请评估未设置

repetition_penalty

的模型，或仅运行生成类任务（

gsm8k

、

aime

）。

Run an evaluation

运行评估

Invoke the evaluator through its Pixi manifest from any working directory:

bash

pixi run --manifest-path "$SKILL_DIR/pixi.toml" eval \
  --model "meta-llama/Llama-3.1-8B-Instruct" \
  --tasks gsm8k

For a quick endpoint and scoring check, lower both limits and use the bundled scorer:

bash

pixi run --manifest-path "$SKILL_DIR/pixi.toml" eval \
  --model "meta-llama/Llama-3.1-8B-Instruct" \
  --tasks gsm8k \
  --seed-limit 4 \
  --full-limit 8 \
  --direct-http

Use

--direct-http

only for connectivity and parsing checks like this one. Its bundled scorer is 0-shot with a custom prompt, so its numbers run far below the few-shot chain-of-thought scores on a model card — in one measured run the same model scored 33% under the direct seed scorer and 82% on the lm-eval full pass. Don't compare

--direct-http

results with published benchmarks; drop the flag for any card comparison so the default lm-eval pass runs instead.

For multiple-choice tasks on an aliased served model, pass the Hugging Face tokenizer ID with

--tokenizer

. This is required, not optional: lm-eval needs a real tokenizer to compute token boundaries, and without it the tokenizer defaults to

--model

and lm-eval fails trying to load the alias as a Hugging Face repo.

bash

pixi run --manifest-path "$SKILL_DIR/pixi.toml" eval \
  --model "my-served-alias" \
  --tokenizer "meta-llama/Llama-3.1-8B-Instruct" \
  --tasks mmlu,hellaswag

The

--model

value must match an ID returned by

/v1/models

--apply-chat-template

wraps each prompt in the model's chat template. Whether to use it depends on your comparison baseline, not just on whether the model is instruction-tuned:

To reproduce published multiple-choice leaderboard scores (MMLU, HellaSwag, ARC, WinoGrande), leave it off — those references are measured on raw completion prompts. Adding it succeeds but produces numbers that can't be compared to the card.
Use it only when you intend to measure the model as it's actually chatted to, and are comparing against a chat-template baseline.

可从任意工作目录通过Pixi配置清单调用评估器：

bash

pixi run --manifest-path "$SKILL_DIR/pixi.toml" eval \
  --model "meta-llama/Llama-3.1-8B-Instruct" \
  --tasks gsm8k

若要快速检查端点与打分功能，可降低两个限制并使用内置打分器：

bash

pixi run --manifest-path "$SKILL_DIR/pixi.toml" eval \
  --model "meta-llama/Llama-3.1-8B-Instruct" \
  --tasks gsm8k \
  --seed-limit 4 \
  --full-limit 8 \
  --direct-http

仅在进行连通性和解析检查时使用

--direct-http

。其内置打分器为0-shot模式并使用自定义提示词，得分远低于模型卡片上的few-shot思维链得分——在某次测试中，同一模型在direct seed打分器下得分为33%，而lm-eval完整测试得分为82%。请勿将

--direct-http

的结果与公开基准对比；若需与模型卡片对比，请移除该参数，使用默认的lm-eval完整测试。

针对使用别名部署的模型运行多选类任务时，请通过

--tokenizer

参数传入Hugging Face分词器ID。此为必填项：lm-eval需要真实的分词器计算token边界，若未传入则默认使用

--model

参数的值，lm-eval会尝试将别名作为Hugging Face仓库加载，导致失败。

bash

pixi run --manifest-path "$SKILL_DIR/pixi.toml" eval \
  --model "my-served-alias" \
  --tokenizer "meta-llama/Llama-3.1-8B-Instruct" \
  --tasks mmlu,hellaswag

--model

参数的值必须与

/v1/models

返回的ID一致。

--apply-chat-template

参数会将每个提示词包裹在模型的聊天模板中。是否使用该参数取决于你的对比基准，而非仅看模型是否为指令微调：

若要复现公开的多选排行榜分数（MMLU、HellaSwag、ARC、WinoGrande），请关闭该参数——这些参考分数是基于原始补全提示词测得的。启用该参数虽能运行，但结果无法与模型卡片对比。
仅当你打算测量模型实际对话时的表现，且对比基准为聊天模板时，才使用该参数。

Interpret the result

结果解读

The command uses these exit codes:

```
0
```
: All tasks completed, or every explicit
```
--target
```
was met.
```
1
```
: Setup, dependency, endpoint, dataset, or lm-eval failure.
```
2
```
: At least one explicit
```
--target
```
was missed.
```
3
```
: A direct seed task scored below its fast-fail floor.

Without

--target

, a valid full run has status

completed

; the evaluator doesn't impose model-independent accuracy targets. A

completed

met

status means the run finished cleanly — it's not by itself a verdict that the model is correct or at parity. To turn a run into a pass/fail or a comparison, supply a baseline yourself.

命令使用以下退出码：

```
0
```
：所有任务完成，或所有指定的
```
--target
```
目标均达成。
```
1
```
：安装、依赖、端点、数据集或lm-eval执行失败。
```
2
```
：至少一个指定的
```
--target
```
目标未达成。
```
3
```
：种子任务得分低于快速失败阈值。

未指定

--target

时，有效的完整运行状态为

completed

；评估器不会设置与模型无关的准确率目标。

completed

或

met

状态仅表示运行顺利完成，并非判定模型正确或达标。若要将运行结果转为通过/失败或进行对比，请自行提供基准值。

Compare against a model card

与模型卡片对比

The evaluator doesn't know any model's expected scores, so a real comparison is a few explicit steps:

Look up the specific metric on the model card or reference, matching the variant this tool produces — for generation tasks that's few-shot chain-of-thought (for example, Llama-3.1-8B-Instruct reports GSM8K around 84.5, 8-shot CoT). Note the shot count and whether a chat template was used.
Pass that number as
```
--target
```
. Be aware
```
--target
```
is a one-sided floor: the run fails (exit
```
2
```
) only when accuracy falls below it, so it gates rather than measuring a two-sided delta.
Read
```
accuracy
```
from the task's summary JSON and report the actual difference from the reference. Allow for sampling noise — the default full run is 200 samples per task (or per subtask for grouped tasks), not the full test set, so treat small gaps as noise rather than regressions. Raise
```
--full-limit
```
toward the full dataset when you need a tighter estimate.

bash

pixi run --manifest-path "$SKILL_DIR/pixi.toml" eval \
  --model "meta-llama/Llama-3.1-8B-Instruct" \
  --tasks gsm8k,mmlu \
  --target "gsm8k:0.84,mmlu:0.68"

Summaries default to

eval-results/<model-name>/

text

eval-results/meta-llama_Llama-3.1-8B-Instruct/
├── eval_summary.json
├── gsm8k_summary.json
└── mmlu_summary.json

Use

--output-dir

to choose another location or

--run-id

to change only the default directory label. The combined summary records the effective CLI configuration and evaluator dependency versions needed to interpret the run.

评估器不知道任何模型的预期得分，因此真实对比需要以下明确步骤：

在模型卡片或参考文档中查找具体指标，确保与本工具生成的变体匹配——对于生成类任务，应为few-shot思维链（例如Llama-3.1-8B-Instruct的GSM8K分数约为84.5，8-shot CoT）。记录shot数和是否使用聊天模板。
将该数值作为
```
--target
```
参数传入。请注意
```
--target
```
是单向下限：仅当准确率低于该值时，运行才会失败（退出码
```
2
```
），因此它用于 gate 而非衡量双向差值。
从任务的汇总JSON中读取
```
accuracy
```
值，报告与参考值的实际差异。考虑采样噪声——默认完整运行每个任务（或分组任务的每个子任务）取200个样本，而非完整测试集，因此小差距视为噪声而非性能退化。若需要更精确的估计，请提高
```
--full-limit
```
至接近完整数据集。

bash

pixi run --manifest-path "$SKILL_DIR/pixi.toml" eval \
  --model "meta-llama/Llama-3.1-8B-Instruct" \
  --tasks gsm8k,mmlu \
  --target "gsm8k:0.84,mmlu:0.68"

汇总结果默认保存至

eval-results/<model-name>/

：

text

eval-results/meta-llama_Llama-3.1-8B-Instruct/
├── eval_summary.json
├── gsm8k_summary.json
└── mmlu_summary.json

可使用

--output-dir

参数选择其他位置，或使用

--run-id

参数修改默认目录名称。合并汇总记录了解析运行所需的有效CLI配置和评估器依赖版本。

Choose tasks

任务选择

Task behavior is as follows:

```
gsm8k
```
and
```
aime
```
: Direct-HTTP seed check, followed by an lm-eval full run. Add
```
--direct-http
```
to use the bundled exact-match scorer for the full run.

mmlu

hellaswag

arc_easy

arc_challenge

winogrande

, and

truthfulqa

: lm-eval full run through

/v1/completions

; require

--enable-echo

```
gpqa
```
: lm-eval multiple-choice run through
```
/v1/completions
```
; requires
```
--enable-echo
```
, and the dataset can require Hugging Face access.
```
babilong
```
: lm-eval chat run; defaults to
```
{"max_seq_lengths": "16k"}
```
metadata.

Each full run defaults to 200 samples (200 per subtask for grouped tasks such as

mmlu

;

gpqa

and

babilong

default to 100). This is a spot check, not the whole test set — raise

--full-limit

when you need a tighter estimate, and expect small run-to-run variation at the default size. The summary reports the aggregate number of effective child samples for grouped tasks.

Only the generation tasks (

gsm8k

aime

) run the direct seed pass; the multiple-choice tasks skip straight to the full run, so seeing no seed accuracy for

mmlu

and friends is expected. The seed's fast-fail floor is a plumbing gate — it catches an unusable endpoint, a malformed request, or broken answer extraction, not normal model-quality variation. To judge quality, use

--target

(see "Compare against a model card"), not the fast-fail floor.

ChartQA isn't registered because stock lm-eval ChartQA prompt formatting isn't compatible with MAX image-token injection. Add a validated MAX-specific task configuration before enabling it.

Print the effective task configuration without installing MAX or starting a server:

bash

pixi run --manifest-path "$SKILL_DIR/pixi.toml" validate

各任务行为如下：

```
gsm8k
```
和
```
aime
```
：先进行Direct-HTTP种子检查，再执行lm-eval完整运行。添加
```
--direct-http
```
参数可在完整运行中使用内置精确匹配打分器。

mmlu

、

hellaswag

、

arc_easy

、

arc_challenge

、

winogrande

和

truthfulqa

：通过

/v1/completions

执行lm-eval完整运行；需要

--enable-echo

参数。

```
gpqa
```
：通过
```
/v1/completions
```
执行lm-eval多选运行；需要
```
--enable-echo
```
参数，且数据集可能需要Hugging Face访问权限。
```
babilong
```
：lm-eval聊天式运行；默认元数据为
```
{"max_seq_lengths": "16k"}
```
。

每个完整运行默认取200个样本（分组任务如

mmlu

的每个子任务取200个样本；

gpqa

和

babilong

默认取100个样本）。这是抽样检查，而非完整测试集——若需要更精确的估计，请提高

--full-limit

，默认样本量下不同运行间可能存在小幅差异。汇总报告会记录分组任务的有效子样本总数。

仅生成类任务（

gsm8k

、

aime

）会运行种子检查；多选类任务直接进入完整运行，因此

mmlu

等任务无种子准确率是正常现象。种子的快速失败阈值是管道检查 gate——用于捕获不可用的端点、格式错误的请求或失效的答案提取，而非正常的模型质量波动。若要判断模型质量，请使用

--target

（见“与模型卡片对比”），而非快速失败阈值。

未注册ChartQA任务，因为标准lm-eval的ChartQA提示词格式与MAX的图像token注入不兼容。请先添加经过验证的MAX专属任务配置，再启用该任务。

无需安装MAX或启动服务器即可打印有效任务配置：

bash

pixi run --manifest-path "$SKILL_DIR/pixi.toml" validate

Evaluate reasoning models

评估推理模型

For a MAX model whose chat template accepts

reasoning_effort

, pass one of

low

medium

high

, or

xhigh

bash

pixi run --manifest-path "$SKILL_DIR/pixi.toml" eval \
  --model "<MODEL_ID>" \
  --tasks gsm8k,aime \
  --reasoning-effort low

The evaluator sends this value through

chat_template_kwargs={"reasoning_effort": ...}

and automatically selects the direct-HTTP full scorer for supported generation tasks. The preflight request fails clearly if the endpoint doesn't accept the field.

对于聊天模板支持

reasoning_effort

的MAX模型，传入

low

、

medium

、

high

或

xhigh

中的一个值：

bash

pixi run --manifest-path "$SKILL_DIR/pixi.toml" eval \
  --model "<MODEL_ID>" \
  --tasks gsm8k,aime \
  --reasoning-effort low

评估器会通过

chat_template_kwargs={"reasoning_effort": ...}

传入该值，并自动为支持的生成类任务选择Direct-HTTP完整打分器。若端点不支持该字段，预检查请求会明确返回失败。

Monitor progress

监控进度

Heartbeat records go to stdout; human-readable logs go to stderr. To archive them, capture each stream to its own file:

bash

pixi run --manifest-path "$SKILL_DIR/pixi.toml" eval \
  --model "<MODEL_ID>" \
  --tasks gsm8k \
  > heartbeats.jsonl \
  2> eval.log

To watch liveness while the run proceeds, keep stdout on a pipe rather than a plain file, so a consumer can read heartbeats as they arrive — the example consumer in the heartbeat reference can't tail a file redirect alone:

bash

pixi run --manifest-path "$SKILL_DIR/pixi.toml" eval \
  --model "<MODEL_ID>" --tasks gsm8k \
  2> eval.log | tee heartbeats.jsonl | python your_consumer.py

Treat a missing heartbeat for twice

--heartbeat-interval

as a possible stall. For reasoning models, a single generation can legitimately exceed the default 30-second interval, so raise

--heartbeat-interval

(and with it the stall threshold) to avoid false alarms on slow-but-healthy runs. The evaluator emits

done

only after writing the corresponding task summary.

心跳记录输出至stdout；人类可读日志输出至stderr。若要存档，请将每个流捕获到单独文件：

bash

pixi run --manifest-path "$SKILL_DIR/pixi.toml" eval \
  --model "<MODEL_ID>" \
  --tasks gsm8k \
  > heartbeats.jsonl \
  2> eval.log

若要在运行过程中监控存活状态，请将stdout保持为管道而非普通文件，以便消费者实时读取心跳——心跳参考文档中的示例消费者无法仅通过文件重定向读取：

bash

pixi run --manifest-path "$SKILL_DIR/pixi.toml" eval \
  --model "<MODEL_ID>" --tasks gsm8k \
  2> eval.log | tee heartbeats.jsonl | python your_consumer.py

若超过两倍

--heartbeat-interval

时间未收到心跳，则视为可能停滞。对于推理模型，单次生成可能合法超过默认30秒间隔，因此请提高

--heartbeat-interval

（以及对应的停滞阈值），避免对缓慢但健康的运行误报。评估器仅在写入对应任务汇总后才会发出

done

信号。

Troubleshoot failures

故障排查

Use these checks:

Connection or model error: Run
```
curl http://localhost:8000/v1/models
```
and pass an exact returned model ID.
Missing prompt logprobs: Restart MAX with
```
--enable-echo
```
(it can't be enabled on a running server); confirm the selected runtime supports log probabilities.
Serving worker crashes on startup with echo and a
```
repetition_penalty
```
tensor-shape error: The served model's
```
generation_config.json
```
sets a non-default
```
repetition_penalty
```
(common in Qwen2.5), so MAX auto-enables a penalty path that echo doesn't support. Use a model whose generation config leaves
```
repetition_penalty
```
unset (Llama-3.1, Qwen3) for multiple-choice tasks, or restrict the run to generation tasks (
```
gsm8k
```
,
```
aime
```
).
```
No module named 'pydantic'
```
from
```
max serve
```
: The serving environment has the slim
```
max
```
package but not
```
max-pipelines
```
. Install
```
max-pipelines
```
.

Missing package: Run

pixi install --manifest-path "$SKILL_DIR/pixi.toml"

again.

Dataset access error: Authenticate with Hugging Face and accept the dataset's terms.
lm-eval task error: List tasks for the installed lm-eval version with
```
pixi run --manifest-path "$SKILL_DIR/pixi.toml" lm-eval ls tasks
```
, then print the effective task configuration with
```
pixi run --manifest-path "$SKILL_DIR/pixi.toml" validate
```
(the
```
validate
```
task runs the script's
```
--validate-thresholds
```
mode).
Out-of-memory or context-length error: Reduce
```
--num-concurrent
```
,
```
--full-limit
```
, or the model's served context length as appropriate.

可使用以下检查步骤：

连接或模型错误：执行
```
curl http://localhost:8000/v1/models
```
，传入返回的精确模型ID。
缺少提示词对数概率：重启MAX并添加
```
--enable-echo
```
参数（无法在运行中的服务器上启用）；确认所选运行时支持对数概率。
启用echo后服务进程启动时因
```
repetition_penalty
```
张量形状错误崩溃：已部署模型的
```
generation_config.json
```
设置了非默认的
```
repetition_penalty
```
（Qwen2.5系列常见），导致MAX自动启用与echo不兼容的惩罚路径。对于多选类任务，请使用未设置
```
repetition_penalty
```
的模型（Llama-3.1、Qwen3），或仅运行生成类任务（
```
gsm8k
```
、
```
aime
```
）。
```
max serve
```
报错
```
No module named 'pydantic'
```
：服务环境安装了精简版
```
max
```
包，但未安装
```
max-pipelines
```
。请安装
```
max-pipelines
```
。

缺少依赖包：再次执行

pixi install --manifest-path "$SKILL_DIR/pixi.toml"

。

数据集访问错误：登录Hugging Face并接受数据集条款。

lm-eval任务错误：使用

pixi run --manifest-path "$SKILL_DIR/pixi.toml" lm-eval ls tasks

查看已安装lm-eval版本支持的任务，再使用

pixi run --manifest-path "$SKILL_DIR/pixi.toml" validate

打印有效任务配置（

validate

任务会以

--validate-thresholds

模式运行脚本）。

内存不足或上下文长度错误：酌情降低
```
--num-concurrent
```
、
```
--full-limit
```
或模型部署的上下文长度。