jetson-llm-serve

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Jetson LLM Serve

Jetson LLM 服务搭建

Encodes the Jetson AI Lab GenAI tutorial: on Orin JetPack 7.2 / L4T r39+, use upstream vLLM 0.20+ (

vllm/vllm-openai:latest

); on older Orin, pick the NVIDIA-AI-IOT prebuilt vLLM container; on Thor, use upstream vLLM 0.20+ or validated native vLLM 0.20+, and use NVIDIA SGLang 26.01 (

nvcr.io/nvidia/sglang:26.01-py3

, SGLang 0.5.5.post2) when SGLang is requested. Set MAXN, make Hugging Face credentials/cache available, and launch an OpenAI-compatible server. Works for both LLMs and VLMs.

本文整合了Jetson AI Lab生成式AI教程的内容：在Orin JetPack 7.2 / L4T r39+上，使用上游vLLM 0.20+（

vllm/vllm-openai:latest

）；在旧版Orin上，选择NVIDIA-AI-IOT预构建的vLLM容器；在Thor上，使用上游vLLM 0.20+或经过验证的原生vLLM 0.20+，当用户要求使用SGLang时，采用NVIDIA SGLang 26.01（

nvcr.io/nvidia/sglang:26.01-py3

，对应SGLang 0.5.5.post2）。设置MAXN模式，配置Hugging Face凭证/缓存，启动兼容OpenAI的服务器。该方案适用于LLM和VLM两种模型。

Purpose

目的

Provide a Jetson-appropriate serving recipe for an LLM or VLM using vLLM or SGLang, including runtime path, launch command, endpoint, and verification step.

为在Jetson上部署LLM或VLM提供适配的服务方案，包括运行路径、启动命令、端点以及验证步骤，支持vLLM或SGLang两种框架。

When to use

使用场景

"Run / serve / host this model on a Jetson."
"Start a vLLM server I can hit from Open WebUI / my app."
After
```
jetson-inference-mem-tune
```
produced launch flags and the user wants to actually start the server.

For recipe-only questions, answer from this document without starting containers. Run live pre-flight checks only when the user asks you to check this device or execute the deployment.

"在Jetson上运行/部署/托管这个模型。"
"启动一个可通过Open WebUI/我的应用访问的vLLM服务器。"
当
```
jetson-inference-mem-tune
```
生成启动参数后，用户需要实际启动服务器时。

仅针对方案类问题时，直接依据本文档作答，无需启动容器。仅当用户要求检查设备或执行部署时，才运行实时预检。

Prerequisites

前置条件

Run on the Jetson host or a shell with Docker access to the Jetson GPU runtime.
Know the target Jetson generation (
```
thor
```
or
```
orin
```
) and the model identifier or local checkpoint path.
Use
```
HF_TOKEN
```
only when the model is gated/private; public models should omit the token environment variable.
Use
```
jetson-inference-mem-tune
```
first when memory headroom or launch flags are uncertain.

在Jetson主机或可访问Jetson GPU运行时的Shell环境中操作。
明确目标Jetson型号（
```
thor
```
或
```
orin
```
）以及模型标识符或本地检查点路径。
仅当模型为受限/私有模型时使用
```
HF_TOKEN
```
；公开模型无需设置该环境变量。
若不确定内存余量或启动参数，先运行
```
jetson-inference-mem-tune
```
。

Instructions

操作步骤

For recipe questions, provide a complete launch recipe instead of trying to call

jetson-llm-serve

as a tool. A complete answer includes:

The Jetson-appropriate runtime path: upstream vLLM 0.20+ (
```
vllm/vllm-openai:latest
```
) or NVIDIA SGLang 26.01 (
```
nvcr.io/nvidia/sglang:26.01-py3
```
, SGLang 0.5.5.post2) on Thor, NVIDIA-AI-IOT vLLM container on older Orin, or upstream vLLM 0.20+ on Orin JetPack 7.2 / L4T r39+.
The model checkpoint / Hugging Face repo the user named.
A
```
docker run
```
+ server command sketch with
```
--host 0.0.0.0 --port 8000
```
.
The OpenAI-compatible endpoint:
```
http://<jetson-ip>:8000/v1
```
.
A verification step such as
```
curl http://localhost:8000/v1/models
```
.

For VLM questions, explicitly say the VLM uses the same vLLM serving flow as an LLM with a different vision-language checkpoint. Do not omit

vLLM

or the Jetson container when answering VLM prompts.

针对方案类问题，提供完整的启动方案，而非尝试调用

jetson-llm-serve

工具。完整回答应包含：

适配Jetson的运行路径：Thor使用上游vLLM 0.20+（
```
vllm/vllm-openai:latest
```
）或NVIDIA SGLang 26.01（
```
nvcr.io/nvidia/sglang:26.01-py3
```
）；旧版Orin使用NVIDIA-AI-IOT vLLM容器；Orin JetPack 7.2 / L4T r39+使用上游vLLM 0.20+（
```
vllm/vllm-openai:latest
```
）。
用户指定的模型检查点/Hugging Face仓库。
包含
```
--host 0.0.0.0 --port 8000
```
的
```
docker run
```
+服务器命令示例。
兼容OpenAI的端点：
```
http://<jetson-ip>:8000/v1
```
。
验证步骤，例如
```
curl http://localhost:8000/v1/models
```
。

针对VLM相关问题，需明确说明VLM采用与LLM相同的vLLM服务流程，仅需更换视觉语言模型检查点。回答VLM相关问题时，不得遗漏

vLLM

或Jetson容器信息。

Step 1 — Pick the runtime path (per Jetson family)

步骤1 — 选择运行路径（按Jetson型号）

Use upstream vLLM 0.20+ on Thor (

vllm/vllm-openai:latest

, or a validated native vLLM 0.20+ install). On Orin JetPack 7.2 / L4T r39+, use upstream vLLM 0.20+ (

vllm/vllm-openai:latest

). On older Orin releases, use the NVIDIA-AI-IOT prebuilt vLLM image (packages) because it ships the correct CUDA / cuDNN / TensorRT stack for that JetPack. Use NVIDIA SGLang 26.01 (

nvcr.io/nvidia/sglang:26.01-py3

, SGLang 0.5.5.post2) on Thor when the user asks for SGLang, RAG, tool-use, or programmable serving; do not recommend native upstream SGLang on Orin unless a JetPack-matched release explicitly supports it.

Jetson family	Runtime path
Thor (T5000, T4000)	upstream vLLM 0.20+ ( `vllm/vllm-openai:latest` ) or NVIDIA SGLang 26.01 ( `nvcr.io/nvidia/sglang:26.01-py3` , SGLang 0.5.5.post2)
AGX Orin / Orin NX / Nano	Orin JetPack 7.2 / L4T r39+: upstream vLLM 0.20+ ( `vllm/vllm-openai:latest` ); older Orin: `ghcr.io/nvidia-ai-iot/vllm:latest-jetson-orin`

To detect the silicon era for image tags:

Source the detector so exports survive in your shell:
bash
```
. skills/jetson-diagnostic/scripts/detect_jetson.sh
```
Check
```
JETSON_GENERATION
```
(
```
thor
```
or
```
orin
```
) and choose the matching runtime path from the table above.
Use
```
JETSON_PRODUCT_LINE
```
for a finer bucket such as
```
thor-agx
```
or
```
orin-nano
```
;
```
JETSON_SKU
```
remains the legacy identifier.

Do not use

bash skills/jetson-diagnostic/scripts/detect_jetson.sh

when you need exported variables in the caller; running with

bash

uses a subshell.

Thor使用上游vLLM 0.20+（

vllm/vllm-openai:latest

）或经过验证的原生vLLM 0.20+安装包。Orin JetPack 7.2 / L4T r39+使用上游vLLM 0.20+（

vllm/vllm-openai:latest

）。旧版Orin需使用NVIDIA-AI-IOT预构建vLLM镜像（包地址），因为它包含适配对应JetPack版本的CUDA/cuDNN/TensorRT栈。当用户要求使用SGLang、RAG、工具调用或可编程服务时，Thor使用NVIDIA SGLang 26.01（

nvcr.io/nvidia/sglang:26.01-py3

，对应SGLang 0.5.5.post2）；除非有明确支持JetPack的版本，否则不推荐在Orin上使用原生上游SGLang。

Jetson型号	运行路径
Thor（T5000、T4000）	上游vLLM 0.20+（ `vllm/vllm-openai:latest` ）或NVIDIA SGLang 26.01（ `nvcr.io/nvidia/sglang:26.01-py3` ，对应SGLang 0.5.5.post2）
AGX Orin / Orin NX / Nano	Orin JetPack 7.2 / L4T r39+：上游vLLM 0.20+（ `vllm/vllm-openai:latest` ）；旧版Orin： `ghcr.io/nvidia-ai-iot/vllm:latest-jetson-orin`

检测芯片世代以选择镜像标签：

加载检测脚本，确保变量在当前Shell中生效：
bash
```
. skills/jetson-diagnostic/scripts/detect_jetson.sh
```
查看
```
JETSON_GENERATION
```
（
```
thor
```
或
```
orin
```
），并从上方表格选择匹配的运行路径。
使用
```
JETSON_PRODUCT_LINE
```
获取更细分的型号，如
```
thor-agx
```
或
```
orin-nano
```
；
```
JETSON_SKU
```
为旧版标识符。

若需要导出变量在当前Shell生效，请勿使用

bash skills/jetson-diagnostic/scripts/detect_jetson.sh

，因为

bash

会启动子Shell，变量无法传递到父Shell。

Step 2 — Set MAXN power mode

步骤2 — 设置MAXN功耗模式

bash

sudo nvpmodel -m 0 && sudo jetson_clocks

Skip this only if the user explicitly asks for a power-constrained run; otherwise benchmark and serving numbers will be inconsistent.

bash

sudo nvpmodel -m 0 && sudo jetson_clocks

仅当用户明确要求受限功耗运行时可跳过此步骤；否则基准测试和服务性能数据会不一致。

Step 3 — Run the server

步骤3 — 启动服务器

On Thor with vLLM, use upstream vLLM 0.20+ (

vllm/vllm-openai:latest

) or a validated native vLLM 0.20+ install:

bash

docker run --rm -it --runtime nvidia --network host --ipc host --name vllm \
  -v "$HOME/.cache/huggingface:/root/.cache/huggingface" \
  -e HF_TOKEN="$HF_TOKEN" \
  vllm/vllm-openai:latest \
  vllm serve <hf-repo-id> \
    --host 0.0.0.0 --port 8000 \
    --max-model-len 8192 \
    --gpu-memory-utilization 0.75 \
    --tensor-parallel-size 1

On Orin JetPack 7.2 / L4T r39+, use upstream vLLM 0.20+ (

vllm/vllm-openai:latest

). On older Orin releases, use the NVIDIA-AI-IOT container:

bash

docker run --rm -it --runtime nvidia --network host --name vllm \
  -v "$HOME/.cache/huggingface:/root/.cache/huggingface" \
  -e HF_TOKEN="$HF_TOKEN" \
  ghcr.io/nvidia-ai-iot/vllm:latest-jetson-orin \
  vllm serve <hf-repo-id> \
    --host 0.0.0.0 --port 8000 \
    --max-model-len 4096 \
    --gpu-memory-utilization 0.85 \
    --tensor-parallel-size 1

HF_TOKEN

is required only for gated/private Hugging Face models; omit the

-e HF_TOKEN="$HF_TOKEN"

line for public models that do not need Hub authentication. Passing

HF_TOKEN

as an environment variable can expose it through Docker inspect output, process metadata, or logs on shared systems. Prefer the narrowest-scoped token possible, rotate/revoke it after shared-container use, and use a mounted credential file or Docker secret when the deployment environment supports that pattern.

Wait for

Application startup complete.

Server is on

http://0.0.0.0:8000/v1

For SGLang on Thor, use NVIDIA SGLang 26.01 (

nvcr.io/nvidia/sglang:26.01-py3

), which packages SGLang 0.5.5.post2 and lists Jetson Thor support. Do not judge Thor SGLang support from older prerelease SGLang results:

bash

docker run --rm -it --runtime nvidia --network host --ipc host --name sglang \
  -v "$HOME/.cache/huggingface:/root/.cache/huggingface" \
  -e HF_TOKEN="$HF_TOKEN" \
  nvcr.io/nvidia/sglang:26.01-py3 \
  python3 -m sglang.launch_server \
    --model-path <hf-repo-id> \
    --host 0.0.0.0 \
    --port 8000 \
    --mem-fraction-static 0.60 \
    --max-running-requests 8

Use SGLang when the user needs RAG/tool-use workflows, structured generation, or SGLang-specific scheduling. For plain high-throughput OpenAI-compatible serving, prefer vLLM unless the user asks for SGLang.

Thor使用vLLM时，采用上游vLLM 0.20+（

vllm/vllm-openai:latest

）或经过验证的原生vLLM 0.20+安装包：

bash

docker run --rm -it --runtime nvidia --network host --ipc host --name vllm \
  -v "$HOME/.cache/huggingface:/root/.cache/huggingface" \
  -e HF_TOKEN="$HF_TOKEN" \
  vllm/vllm-openai:latest \
  vllm serve <hf-repo-id> \
    --host 0.0.0.0 --port 8000 \
    --max-model-len 8192 \
    --gpu-memory-utilization 0.75 \
    --tensor-parallel-size 1

Orin JetPack 7.2 / L4T r39+使用上游vLLM 0.20+（

vllm/vllm-openai:latest

）。旧版Orin使用NVIDIA-AI-IOT容器：

bash

docker run --rm -it --runtime nvidia --network host --name vllm \
  -v "$HOME/.cache/huggingface:/root/.cache/huggingface" \
  -e HF_TOKEN="$HF_TOKEN" \
  ghcr.io/nvidia-ai-iot/vllm:latest-jetson-orin \
  vllm serve <hf-repo-id> \
    --host 0.0.0.0 --port 8000 \
    --max-model-len 4096 \
    --gpu-memory-utilization 0.85 \
    --tensor-parallel-size 1

仅当Hugging Face模型为受限/私有模型时需要

HF_TOKEN

；公开模型无需添加

-e HF_TOKEN="$HF_TOKEN"

行。将

HF_TOKEN

作为环境变量传递可能会通过Docker inspect输出、进程元数据或日志泄露到共享系统中。尽量使用权限范围最小的令牌，在共享容器使用后轮换/撤销令牌；若部署环境支持，优先使用挂载凭证文件或Docker密钥的方式。

等待出现

Application startup complete.

提示，服务器将运行在

http://0.0.0.0:8000/v1

。

Thor使用SGLang时，采用NVIDIA SGLang 26.01（

nvcr.io/nvidia/sglang:26.01-py3

），该版本包含SGLang 0.5.5.post2并支持Jetson Thor。请勿依据旧版预发布SGLang的结果判断Thor的SGLang支持情况：

bash

docker run --rm -it --runtime nvidia --network host --ipc host --name sglang \
  -v "$HOME/.cache/huggingface:/root/.cache/huggingface" \
  -e HF_TOKEN="$HF_TOKEN" \
  nvcr.io/nvidia/sglang:26.01-py3 \
  python3 -m sglang.launch_server \
    --model-path <hf-repo-id> \
    --host 0.0.0.0 \
    --port 8000 \
    --mem-fraction-static 0.60 \
    --max-running-requests 8

当用户需要RAG/工具调用工作流、结构化生成或SGLang专属调度时使用SGLang。若仅需高吞吐量的OpenAI兼容服务，优先选择vLLM，除非用户明确要求使用SGLang。

SKU-appropriate defaults

适配型号的默认参数

Knob	Orin Nano / NX	AGX Orin / Thor
`--max-model-len`	`4096`	`8192`
`--gpu-memory-utilization`	`0.85`	`0.85`
`--tensor-parallel-size`	`1`	`1`

If the server OOMs at startup, lower

--gpu-memory-utilization

by 0.05 and re-launch (or run

jetson-inference-mem-tune

for a workload-aware recommendation).

参数	Orin Nano / NX	AGX Orin / Thor
`--max-model-len`	`4096`	`8192`
`--gpu-memory-utilization`	`0.85`	`0.85`
`--tensor-parallel-size`	`1`	`1`

若服务器启动时出现内存不足（OOM），将

--gpu-memory-utilization

降低0.05后重新启动（或运行

jetson-inference-mem-tune

获取适配工作负载的建议值）。

Quantization preferences (matters more than the runtime)

量化偏好（比运行时更重要）

For vLLM and SGLang, choose checkpoint formats by Jetson family:

Jetson family	First choice	Acceptable fallback
Thor	NVFP4 when the model/runtime supports it	W4A16
Orin Nano / NX	W4A16	AWQ or GPTQ 4-bit
AGX Orin	W4A16	AWQ or GPTQ 4-bit

For llama.cpp and

Ollama

, use GGUF model quantization names instead: recommend INT4 / Q4_K_M GGUF on both Orin and Thor, and choose a smaller INT4 GGUF model if memory is tight. Do not call GGUF Q4_K_M a W4A16/AWQ/GPTQ model. NVFP4 is Thor-preferred and Thor-tuned for runtimes that support it.

针对vLLM和SGLang，按Jetson型号选择检查点格式：

Jetson型号	首选方案	可接受的备选方案
Thor	NVFP4（若模型/运行时支持）	W4A16
Orin Nano / NX	W4A16	AWQ或GPTQ 4-bit
AGX Orin	W4A16	AWQ或GPTQ 4-bit

针对llama.cpp和

Ollama

，使用GGUF模型量化名称：推荐在Orin和Thor上使用INT4 / Q4_K_M GGUF；若内存紧张，选择更小的INT4 GGUF模型。请勿将GGUF Q4_K_M称为W4A16/AWQ/GPTQ模型。NVFP4是Thor的首选方案，且针对支持的运行时做了调优。

VLM mode

VLM模式

VLMs use the same flow as LLMs: same container, same

vllm serve

invocation, different vision-language checkpoint. The container handles image preprocessing. For a VLM-specific browser UI, use the

live-vlm-webui

container; for a generic chat UI for either, use Open WebUI pointed at

http://<jetson-ip>:8000/v1

VLM采用与LLM相同的流程：相同的容器、相同的

vllm serve

调用命令，仅需更换视觉语言模型检查点。容器会处理图像预处理。若需要VLM专属浏览器UI，使用

live-vlm-webui

容器；若需要通用聊天UI，将Open WebUI指向

http://<jetson-ip>:8000/v1

即可。

Do not fabricate device capacity

请勿虚构设备能力

Do not invent RAM totals, free-memory values, model sizes, JetPack versions, or SKU/variant names when giving a serving recipe. If capacity matters, either run the live pre-flight checks (when execution is allowed) or hand off to

jetson-inference-mem-tune

jetson-memory-audit

. If live data is not available, say the value is unknown and provide conservative defaults instead of quoting a made-up number.

提供服务方案时，请勿编造内存总量、可用内存值、模型大小、JetPack版本或型号名称。若需要设备能力信息，要么运行实时预检（允许执行操作时），要么转至

jetson-inference-mem-tune

jetson-memory-audit

。若无法获取实时数据，说明数值未知并提供保守默认值，而非编造数据。

Pre-flight checklist (the agent should verify before running Step 3)

预检清单（执行步骤3前，Agent需验证）

On a Jetson (
```
/proc/device-tree/model
```
contains
```
NVIDIA Jetson
```
).
```
nvpmodel -q
```
reports a recognized max-performance mode:
```
MAXN
```
or
```
MAXN_*
```
such as
```
MAXN_SUPER
```
. Wattage-named modes should be reported as warnings unless the user explicitly confirms they are the intended benchmark mode for that device.
On Thor, check whether MIG is enabled before launching (
```
nvidia-smi -L
```
and
```
nvidia-smi mig -lgi
```
). If MIG is enabled, warn that vLLM/SGLang may see only a MIG slice or no CUDA device.
On Thor with MIG or display/camera contention, inspect GPU users with
```
sudo lsof /dev/nvidia*
```
. Display managers,
```
Xorg
```
/GNOME, or
```
nvargus-daemon
```
may hold GPU device files; do not stop services or change MIG mode unless the user explicitly approves.
No container named
```
vllm
```
already running (
```
docker ps --format '{{.Names}}'
```
); otherwise
```
docker rm -f vllm
```
first.
Docker exposes the NVIDIA runtime (
```
docker info | grep -i 'runtimes.*nvidia'
```
), or a GPU-enabled container can run
```
nvidia-smi
```
.
```
~/.cache/huggingface
```
exists;
```
HF_TOKEN
```
is set if the model is gated.

运行环境为Jetson（
```
/proc/device-tree/model
```
包含
```
NVIDIA Jetson
```
）。
```
nvpmodel -q
```
显示已识别的最高性能模式：
```
MAXN
```
或
```
MAXN_*
```
（如
```
MAXN_SUPER
```
）。除非用户明确确认是该设备的预期基准测试模式，否则需对以功耗命名的模式发出警告。
在Thor上，启动前检查是否启用了MIG（
```
nvidia-smi -L
```
和
```
nvidia-smi mig -lgi
```
）。若已启用MIG，需警告vLLM/SGLang可能只能看到MIG分片或无法识别CUDA设备。
在Thor上，若存在MIG或显示/相机资源竞争，使用
```
sudo lsof /dev/nvidia*
```
检查GPU使用者。显示管理器、
```
Xorg
```
/GNOME或
```
nvargus-daemon
```
可能占用GPU设备文件；除非用户明确批准，否则请勿停止服务或更改MIG模式。
没有名为
```
vllm
```
的容器正在运行（
```
docker ps --format '{{.Names}}'
```
）；否则先执行
```
docker rm -f vllm
```
。
Docker已暴露NVIDIA运行时（
```
docker info | grep -i 'runtimes.*nvidia'
```
），或GPU容器可运行
```
nvidia-smi
```
。
```
~/.cache/huggingface
```
已存在；若模型为受限模型，已设置
```
HF_TOKEN
```
。

Limitations

限制

This skill provides serving commands and pre-flight checks; it does not benchmark the deployed server.
Container tags such as
```
latest
```
are mutable. For release or compliance deployments, pin a digest and record it with the deployment notes.
vLLM and SGLang memory limits still depend on model architecture, quantization, context length, and concurrent request count. Use
```
jetson-inference-mem-tune
```
when a command OOMs or memory headroom matters.
Thor vLLM requires upstream vLLM 0.20+ or newer. Older upstream vLLM images may not support Thor / SM 11.0 correctly.
Thor SGLang should use NVIDIA SGLang 26.01 or newer release notes that explicitly list Jetson Thor support. NVIDIA SGLang 26.01 contains SGLang 0.5.5.post2.
On Thor, MIG, desktop display, or camera services can hide the full GPU from containers. This skill should detect and warn only; disabling MIG or stopping services such as
```
gdm3
```
or
```
nvargus-daemon
```
requires explicit user approval.
Host-native vLLM/SGLang on Thor should be used only when that install is already validated on the target JetPack.

本方案仅提供服务命令和预检步骤，不包含已部署服务器的基准测试。
如
```
latest
```
这类容器标签是可变的。针对发布或合规部署，需固定镜像摘要并记录在部署文档中。
vLLM和SGLang的内存限制仍取决于模型架构、量化方式、上下文长度和并发请求数。若命令出现OOM或内存受限，使用
```
jetson-inference-mem-tune
```
。
Thor上的vLLM需要上游vLLM 0.20+或更高版本。旧版上游vLLM镜像可能无法正确支持Thor/SM 11.0。
Thor上的SGLang应使用NVIDIA SGLang 26.01或更高版本（发布说明需明确支持Jetson Thor）。NVIDIA SGLang 26.01包含SGLang 0.5.5.post2。
在Thor上，MIG、桌面显示或相机服务可能会隐藏完整GPU资源，导致容器无法访问。本方案仅负责检测并发出警告；禁用MIG或停止
```
gdm3
```
/
```
nvargus-daemon
```
等服务需获得用户明确批准。
Thor上的主机原生vLLM/SGLang仅在已针对目标JetPack版本验证通过的情况下使用。

Hand off to

Source

来源

Jetson AI Lab — Introduction to GenAI on Jetson: How to Run LLMs and VLMs and NVIDIA-AI-IOT GHCR packages.

Jetson AI Lab — Jetson生成式AI入门：如何运行LLM和VLM 以及 NVIDIA-AI-IOT GHCR包。

jetson-llm-serve

Original

Translation

Jetson LLM Serve

Jetson LLM 服务搭建

Purpose

目的

When to use

使用场景

Prerequisites

前置条件

Instructions

操作步骤

Step 1 — Pick the runtime path (per Jetson family)

步骤1 — 选择运行路径（按Jetson型号）

Step 2 — Set MAXN power mode

步骤2 — 设置MAXN功耗模式

Step 3 — Run the server

步骤3 — 启动服务器

SKU-appropriate defaults

适配型号的默认参数

Quantization preferences (matters more than the runtime)

量化偏好（比运行时更重要）

VLM mode

VLM模式

Do not fabricate device capacity

请勿虚构设备能力

Pre-flight checklist (the agent should verify before running Step 3)

预检清单（执行步骤3前，Agent需验证）

Limitations

限制

Hand off to

相关工具

Source

来源