local-model-triage

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Local model triage

本地模型问题排查

A local model "acting dumb" in a harness has exactly three fault domains. Check them in this order — cheapest first.

当本地模型在Harness中“表现失常”时，故障来源恰好分为三类。按以下顺序排查——优先排查成本最低的环节。

1. Serving config (most common)

1. 服务配置（最常见）

Context window: serving defaults are often 4-8k; agent harnesses need 32k+. llama.cpp:
```
--ctx-size
```
; vLLM:
```
--max-model-len
```
; ollama:
```
num_ctx
```
in the Modelfile or request. Symptom: the agent forgets earlier turns or tool schemas mid-session.
Tool-call parsing: agentic use requires the server's native tool-call template. llama.cpp:
```
--jinja
```
; vLLM:
```
--enable-auto-tool-choice --tool-call-parser <family>
```
. Symptom: tool calls arrive as prose JSON in the text channel instead of structured calls.
Quantization too aggressive: below ~Q4, tool-call reliability degrades before chat quality does. Symptom: schema-ish but malformed arguments.

上下文窗口：服务默认值通常为4-8k；而Agent harness需要32k以上。llama.cpp：使用
```
--ctx-size
```
参数；vLLM：使用
```
--max-model-len
```
参数；ollama：在Modelfile或请求中设置
```
num_ctx
```
。症状：代理在会话中途忘记之前的对话回合或工具 schema。
工具调用解析：智能代理场景需要服务器原生的工具调用模板。llama.cpp：使用
```
--jinja
```
参数；vLLM：使用
```
--enable-auto-tool-choice --tool-call-parser <family>
```
参数。症状：工具调用以散文式JSON形式出现在文本通道中，而非结构化调用。
量化过度：当量化级别低于约Q4时，工具调用的可靠性会先于聊天质量下降。症状：参数类似schema格式但存在语法错误。

2. Model tier

2. 模型层级

Run

packages/local-models/probe.sh

; compare the served model against

packages/local-models/MODELS.md

for the hardware tier. A model below the matrix's "agentic-capable" line will loop, ignore system prompts, or hallucinate tool names — no serving flag fixes that; move up a tier or to a coder/agentic variant.

运行

packages/local-models/probe.sh

脚本；将部署的模型与

packages/local-models/MODELS.md

中的硬件层级进行对比。低于矩阵中“具备智能代理能力”线的模型会出现循环、忽略系统提示或虚构工具名称的问题——这类问题无法通过服务参数修复，需要升级模型层级或更换为编码/智能代理专用变体。

3. Harness wiring

3. Harness连接

Endpoint alive?

curl -s http://localhost:<port>/v1/models

Model id in the harness config must EXACTLY match the served id (
```
pi
```
:
```
~/.pi/agent/models.json
```
; opencode: provider
```
models
```
key; hermes:
```
model:
```
in
```
~/.hermes/cli-config.yaml
```
).
Streaming quirks: if output truncates only in one harness, test the same prompt via curl; if curl is fine, the fault is that harness's provider config, not the model.

Report findings as: fault domain → evidence → one-line fix.

端点是否正常？执行

curl -s http://localhost:<port>/v1/models

检查。

Harness配置中的模型ID必须与部署的模型ID完全匹配（pi：
```
~/.pi/agent/models.json
```
；opencode：provider的
```
models
```
键；hermes：
```
~/.hermes/cli-config.yaml
```
中的
```
model:
```
字段）。
流式传输异常：如果仅在某个Harness中出现输出截断，通过curl测试相同的提示；如果curl测试正常，则故障出在该Harness的提供商配置，而非模型本身。

报告结果格式：故障领域 → 证据 → 一行式修复方案。