endpoint-ops

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Endpoint ops

端点运维

You operate a Serverless endpoint that already exists. Everything starts by reading its current state (the endpoint config, its health, its workers, its logs) before you say anything or change anything. Diagnosis is always read-only. You change a setting only when the user asked for a change, and then only the exact setting agreed.

One property of the Runpod update path drives the central rule here: an endpoint update is a PATCH. Only the fields present in the body change and omitted fields are left untouched, so you send exactly the setting you were asked to change and nothing else. The one exception is the GPU selection:

gpu.pools

and

gpu.excludedTypes

are a single selection, so a PATCH that sends

pools

without

excludedTypes

clears the exclusions. When you touch either, send both. What the PATCH does not give you is proof: its own response is not a read. Read the config first for the before value, and read it back afterward for the after value.

你将操作一个已存在的Serverless端点。在给出任何回复或执行任何修改前，所有操作都从读取其当前状态开始（包括端点配置、健康状态、worker列表、日志）。诊断过程始终为只读模式。仅当用户明确要求修改时，才会更改设置，且仅修改双方确认的精确项。

Runpod更新路径的一个特性决定了这里的核心规则：端点更新采用PATCH方式。只有请求体中包含的字段会被修改，未提及的字段将保持不变，因此你只需发送用户要求修改的设置，不要额外添加其他字段。唯一的例外是GPU选型：

gpu.pools

和

gpu.excludedTypes

属于同一个配置项，因此如果PATCH请求只发送

pools

而不包含

excludedTypes

，会清空已有的排除列表。当你修改其中任意一个时，必须同时发送两个字段。PATCH请求的响应不能作为修改生效的证明：它本身的响应不是读取操作。请先读取配置获取修改前的值，修改后再回读配置确认修改后的值。

Required capabilities

必备能力

Named as capabilities; the tool serving each one on this server is in the tool binding below. This skill reads before it mutates, and mutates only what the user authorized.

List and inspect endpoints: resolve "which endpoint" and read the full current config.
Read endpoint health and list workers: the stuck-queue / dying-worker evidence.
Read a worker's logs: the crash / model-load failure line.
Read GPU stock per data center, with the CUDA versions that have capacity: is a stuck queue a capacity problem?
Update an endpoint (worker min/max, idle timeout, scaler, FlashBoot, disk, env): the gated tune.
Pin GPUs / set GPU count: the pool plus
```
excludedTypes
```
selection and
```
gpu.count
```
;
```
set-endpoint-gpus
```
builds it and preserves the other settings.
Read release history: which build the workers are running.
Cancel one job; purge the queue. These are the queue-management actions, each gated.
Submit and poll a job to re-verify after a change.

以下为各项能力名称；本服务器上对应每项能力的工具见下方的工具绑定表。本技能遵循“先读取后修改”原则，且仅修改用户授权的内容。

列出与检查端点：确定“目标端点”，并读取完整的当前配置。
读取端点健康状态与列出worker：获取队列卡住/worker异常退出的证据。
读取worker日志：找到崩溃/模型加载失败的关键日志行。
读取各数据中心的GPU库存，以及有可用容量的CUDA版本：判断队列卡住是否是容量问题。
更新端点配置（worker最小/最大数量、空闲超时、扩缩容器、FlashBoot、磁盘、环境变量）：受控的调优操作。
固定GPU/设置GPU数量：包括资源池加
```
excludedTypes
```
选型以及
```
gpu.count
```
；
```
set-endpoint-gpus
```
工具会自动构建该配置并保留其他设置不变。
读取发布历史：查看worker当前运行的构建版本。
取消单个作业；清空队列。这些是队列管理操作，每项都需要授权。
提交并轮询作业，用于修改后的重新验证。

Phase 1 — diagnose (always, read-only)

阶段1 — 诊断（始终为只读模式）

Resolve the endpoint (list endpoints and match by name if needed), read its config, then read health and workers before forming a conclusion. Classify against the symptom:

Jobs stuck
IN_QUEUE
. Read health and the worker list. If max workers is 0 or all workers are throttled/unhealthy, the queue has nothing to run on. An endpoint idle for 3 days has
```
workers.max
```
lowered to 2 and after 7 days to 0, and it stays there until raised, so a long-idle endpoint "stuck" on its first job is usually this. Separate the causes: worker limit (max-workers too low or lowered by idleness, or min 0 with a cold scaler) vs capacity (workers can't acquire the selected GPU:
```
throttled
```
above 0; cross-check that GPU type's availability with
```
product:["SERVERLESS"]
```
, and its CUDA versions with capacity when the endpoint pins a CUDA floor) vs crash loop (workers cycling: read their logs with
```
source=system
```
; a container that exits before the handler runs can still report RUNNING or INITIALIZING while the job stays queued, so the logs settle it, not the worker status). Name which one, quoting the health/worker numbers you read.
Workers crash-looping. List workers, find the failing ones, read their logs, and quote the decisive line: a missing model file, a gated 403, an OOM, a bad handler import. Report what the log says, not a guess. OOM → the GPU is undersized for the real memory pattern; a load/import error → the image or env is wrong.
Cold starts hurt. Read the config. The levers, in order: enable FlashBoot; raise the idle timeout so warm workers linger between bursts; set a nonzero min-worker to keep capacity warm (state that a min-worker bills around the clock; it is an always-on worker); move the model load off the boot with model caching (network volume, Runpod's model repository, or weights baked into the image); and make sure the scaler isn't scaling to zero between every request. Name the levers and their cost before touching anything.
Load-balancer endpoint, workers never ready. On a load-balancer (custom-HTTP) endpoint the load balancer decides readiness by polling the worker's health endpoint:
```
HEALTH_CHECK_PATH
```
(default
```
/ping
```
) on
```
PORT_HEALTH
```
(default 80), which is separate from the serving
```
PORT
```
(default 80). Workers stuck un-ready while requests arrive usually means nothing answers that path on that port; the endpoint read gives you the exact health URL being polled. The fix is a redeploy that exposes the port and serves the health path (see
```
serverless-deploy
```
), not a scaler change.

End phase 1 with the diagnosis and the recommended change, but do not apply it unless the prompt authorized a change. If the ask was only "why / what's wrong", you are done here.

先定位目标端点（列出所有端点，必要时按名称匹配），读取其配置，再读取健康状态和worker列表，之后再得出结论。根据症状进行分类：

作业卡在
IN_QUEUE
状态。读取健康状态和worker列表。如果最大worker数为0，或所有worker都被限流/不健康，队列就没有可用的运行资源。空闲3天的端点，
```
workers.max
```
会被下调至2；空闲7天后会下调至0，且会一直保持该值直到被手动调高，因此长期空闲的端点在第一个作业时“卡住”通常是这个原因。区分以下原因：worker数量限制（最大worker数过低或因空闲被下调，或最小worker数为0且使用冷启动扩缩容器）、容量不足（worker无法获取所选GPU：
```
throttled
```
值大于0；可通过
```
product:["SERVERLESS"]
```
交叉核对该GPU类型的可用性，如果端点设置了CUDA最低版本，还需核对对应有容量的CUDA版本）、崩溃循环（worker反复重启：使用
```
source=system
```
读取其日志；在handler运行前就退出的容器仍可能显示RUNNING或INITIALIZING状态，而作业一直处于排队状态，因此应以日志为准，而非worker状态）。请明确指出是哪种原因，并引用你读取到的健康/worker数值。
worker崩溃循环。 列出所有worker，找到故障的worker，读取其日志，并引用关键的错误行：比如模型文件缺失、权限受限的403错误、OOM（内存不足）、handler导入错误。请报告日志的实际内容，而非猜测。OOM → GPU显存不足以支撑实际的内存使用模式；加载/导入错误 → 镜像或环境变量配置有误。
冷启动影响大。 读取配置。可调整的手段按优先级排序为：启用FlashBoot；提高空闲超时时间，让预热的worker在流量峰值间隔期保持存活；设置非零的最小worker数以保持容量预热（需说明最小worker会按全天计费，相当于常驻worker）；通过模型缓存（网络卷、Runpod模型仓库或内置权重的镜像）将模型加载从启动流程中剥离；确保扩缩容器不会在每次请求之间都缩容到零。在进行任何调整前，请先说明所有可选手段及其成本。
负载均衡器端点的worker始终未就绪。 对于负载均衡器（自定义HTTP）端点，负载均衡器通过轮询worker的健康端点来判断就绪状态：健康端点为
```
PORT_HEALTH
```
（默认80）上的
```
HEALTH_CHECK_PATH
```
（默认
```
/ping
```
），该端口与提供服务的
```
PORT
```
（默认80）是分开的。请求到达时worker一直处于未就绪状态，通常意味着该端口上的路径没有响应；读取端点配置可以获取被轮询的准确健康URL。解决方法是重新部署，暴露对应端口并提供健康路径（参见
```
serverless-deploy
```
），而非调整扩缩容器。

阶段1结束时给出诊断结果和建议的修改方案，但除非用户提示授权修改，否则不要执行。如果用户的问题只是“为什么/出了什么问题”，到这里就完成了。

Phase 2 — tune safely (only if the user authorized a change)

阶段2 — 安全调优（仅在用户授权修改时执行）

Advance only when the prompt asks to change a named endpoint's setting.

The read-back invariant. Read the endpoint's full config first, so you hold the before values. Apply the change as a PATCH carrying only the fields the user authorized. Sending fields you were not asked about is how you overwrite a setting they tuned by hand. Then read the config back and confirm the target field holds its new value and the neighbours you quoted are unchanged.
Autoscaling tune, bursty traffic. For bursty workloads, tune the scaler and worker band: raise max workers for the burst ceiling, set the queue-delay or request-count scaler target to how fast you want to absorb the burst, and set idle timeout to how long to hold warm workers between waves (a queue endpoint scaling on request count rejects
```
workers.idleTimeout
```
, and the account's worker quota can refuse a higher
```
workers.max
```
). Change only the fields the user agreed; state the cost of a higher min-worker.
Pin to one GPU type. When the user wants the endpoint locked to one exact card, select the pool that contains the card and list the pool's other GPU types in
```
gpu.excludedTypes
```
(at least one type must remain), with the CUDA floor if given.
```
set-endpoint-gpus
```
builds that exclusion list for you and re-sends the endpoint's other settings unchanged;
```
update-endpoint
```
can carry the same
```
gpu
```
object, but there
```
pools
```
and
```
excludedTypes
```
are one selection, so send both together.
```
allowedCudaVersions
```
and
```
minCudaVersion
```
are mutually exclusive (400 if both are sent). Changing the CUDA constraint or the pool does not evict running workers; bounce
```
workers.max
```
to 0 and back when the change must apply now. Read the endpoint back after a pin and confirm the scaling settings survived.

仅当用户提示要求修改某个指定的端点设置时，才进入此阶段。

回读不变原则。 先读取端点的完整配置，获取修改前的数值。以PATCH方式提交修改，请求体中仅包含用户授权的字段。发送未被要求修改的字段会覆盖用户手动调整过的设置。修改完成后回读配置，确认目标字段已更新为新值，且你之前提及的相邻字段未被改动。
自动扩缩容调优，应对突发流量。 对于突发型工作负载，调整扩缩容器和worker数量范围：提高最大worker数以应对流量峰值上限，设置队列延迟或请求数扩缩容目标值来控制吸收突发流量的速度，设置空闲超时时间来决定流量波峰之间保留预热worker的时长（基于请求数扩缩容的队列端点不支持
```
workers.idleTimeout
```
，且账户的worker配额可能会限制更高的
```
workers.max
```
）。仅修改用户同意的字段；并说明提高最小worker数的成本。
固定为单一GPU类型。 当用户希望端点锁定到某一具体型号的显卡时，选择包含该显卡的资源池，并将该资源池中的其他GPU类型列入
```
gpu.excludedTypes
```
（至少需保留一种类型），如果有CUDA最低版本要求也一并设置。
```
set-endpoint-gpus
```
工具会自动为你构建排除列表，并重发端点的其他设置保持不变；
```
update-endpoint
```
也可以携带相同的
```
gpu
```
对象，但在该接口中
```
pools
```
和
```
excludedTypes
```
属于同一个配置项，因此必须同时发送。
```
allowedCudaVersions
```
和
```
minCudaVersion
```
是互斥的（同时发送会返回400错误）。修改CUDA约束或资源池不会驱逐正在运行的worker；如果需要立即生效，可将
```
workers.max
```
先调到0再调回原值。固定GPU后回读端点配置，确认扩缩容设置未被改动。

Phase 3 — manage the queue (gated actions)

阶段3 — 管理队列（受控操作）

Cancel one job. When the user names a job to cancel, cancel that job by id. Cancel affects only queued/in-progress jobs. After the cancel, VERIFY with a fresh
```
get-job-status
```
read. The cancel call's own response is not verification (an unread write is an unverified write); the final answer reports the re-read state. If the re-read shows the job already reached a terminal state before your cancel, say exactly that; it is a complete, honest outcome.
Purge the queue. When the user wants every pending job cleared, purge the queue. This removes all pending jobs; in-progress jobs keep running. Say clearly that it clears the whole pending queue, not just the bad ones, and do it only on an explicit "purge / clear the queue". Re-read health afterward to confirm the queue drained.

取消单个作业。 当用户指定要取消的作业时，通过ID取消该作业。取消操作仅影响排队中/进行中的作业。取消后，通过新的
```
get-job-status
```
读取操作进行验证。取消调用本身的响应不能作为验证依据（未经过读取的写入是未验证的写入）；最终答复需报告重新读取的状态。如果重新读取显示作业在你取消前已经达到终态，请如实说明；这是完整、诚实的结果。
清空队列。 当用户希望清除所有待处理作业时，清空队列。此操作会移除所有待处理作业；正在进行的作业会继续运行。请明确说明这会清空整个待处理队列，而非仅清除有问题的作业，且仅在用户明确要求“purge / 清空队列”时执行。之后重新读取健康状态，确认队列已清空。

Hard rules

硬性规则

Every settings change is read → update → read: (1) read the current config, (2) apply the update, (3) a FRESH
```
get-endpoint
```
read AFTER it. The before/after values the final answer quotes come from reads (1) and (3). The update call's own response is not a read; never claim a read-back you did not perform.
Change only the setting the user authorized, on the endpoint they named. Never tune, pin, or purge an endpoint the user did not name.
Diagnosis is read-only, and read-only includes ZERO job submissions. Never fire a
```
run
```
/
```
runsync
```
"test probe" while diagnosing: a submitted job bills compute and changes the queue, so it is a mutation, and on a broken endpoint it burns money to reproduce what the worker states already show. Diagnose from the endpoint/health/worker/log reads only; a recommended fix is presented, not applied, unless the prompt authorizes it.
A queue purge clears the entire pending queue. Say so before doing it; never purge on a vague "clean this up".
State the cost of any setting that bills continuously (a nonzero min-worker keeps workers warm and billed) before applying it.
Quote the health/worker/log field you read. No "probably" about a number the tool returns.
Never cancel or purge jobs the user did not ask you to; do not "helpfully" clear a queue during a diagnosis.
If a read (health, workers, logs) fails, report it and stop; do not guess the cause.

所有设置修改都遵循“读取→更新→读取”流程：(1) 读取当前配置，(2) 执行更新，(3) 更新后进行一次全新的
```
get-endpoint
```
读取。最终答复中引用的修改前/后值来自第(1)和第(3)步的读取结果。更新调用本身的响应不是读取结果；绝不要声称执行了你并未做的回读操作。
仅修改用户授权的、指定端点上的设置。绝不对用户未提及的端点进行调优、固定GPU或清空队列操作。
诊断是只读的，只读包括不提交任何作业。诊断期间绝不要发起
```
run
```
/
```
runsync
```
“测试探测”：提交的作业会产生计算费用并改变队列状态，因此属于修改操作，而且在故障端点上，这只是浪费钱去复现worker状态已经显示的问题。仅通过端点/健康/worker/日志的读取结果进行诊断；仅给出建议的修复方案，不执行，除非用户提示授权。
队列清空会清除整个待处理队列。执行前必须说明这一点；绝不要因为模糊的“清理一下”就执行清空。
对于任何会持续计费的设置（非零的最小worker数会保持worker预热并产生费用），在应用前必须说明其成本。
引用你读取到的健康/worker/日志字段。对于工具返回的数值，不要使用“可能”之类的模糊表述。
绝不要取消或清空用户未要求的作业；不要在诊断期间“好心”清理队列。
如果读取操作（健康、worker、日志）失败，报告失败并停止操作；不要猜测原因。

Error handling

错误处理

A setting nobody touched comes back changed → it was not the PATCH, which leaves omitted fields untouched (except
```
gpu.pools
```
sent without
```
excludedTypes
```
, which clears the exclusions). Look at the other write paths: a GPU pin that re-sent the whole config, a template save, a redeploy, or a concurrent change by someone else. Re-read before re-applying anything.
```
IN_QUEUE
```
with ready workers and nothing in progress → do not wait it out: read the worker logs (
```
source=system
```
); a container that exits before the handler runs looks exactly like this and points at a broken image, not at the scaler.
Worker logs show OOM → the GPU is undersized; the fix is a bigger GPU (re-deploy via
```
serverless-deploy
```
), not a scaler tweak.
Cancel/purge returns success but health still shows queued jobs → re-read after a moment; the queue count settles slightly after the action.
```
get-job-status
```
answers 404 for a job id that was valid → the job's TTL (24 h from submission by default) expired and removed it; it is not a wrong id.
A GPU-pin request when
```
set-endpoint-gpus
```
is missing from the session → make the same selection through
```
update-endpoint
```
, sending
```
gpu.pools
```
and
```
gpu.excludedTypes
```
together.

未被修改的设置返回时发生了变化 → 这不是PATCH导致的，因为PATCH不会改动未提及的字段（除非发送了
```
gpu.pools
```
但没带
```
excludedTypes
```
，会清空排除列表）。请检查其他写入路径：比如重发了整个配置的GPU固定操作、模板保存、重新部署，或者其他人的并发修改。在重新应用任何设置前，请先重新读取。
有就绪的worker但作业仍处于
```
IN_QUEUE
```
状态，且没有进行中的作业 → 不要等待：读取worker日志（
```
source=system
```
）；在handler运行前容器就退出的情况会呈现完全相同的特征，说明镜像有问题，而非扩缩容器的问题。
worker日志显示OOM → GPU显存不足；解决方法是使用更大的GPU（通过
```
serverless-deploy
```
重新部署），而非调整扩缩容器。
取消/清空返回成功，但健康状态仍显示有排队作业 → 稍等片刻后重新读取；队列计数会在操作后稍晚才更新稳定。
对一个有效的作业ID，
```
get-job-status
```
返回404 → 作业的TTL（默认提交后24小时）到期，已被移除；不是ID错误。
收到固定GPU的请求，但会话中没有
```
set-endpoint-gpus
```
工具 → 通过
```
update-endpoint
```
完成相同的选型，同时发送
```
gpu.pools
```
和
```
gpu.excludedTypes
```
。

Tool binding

工具绑定

Capability	Tool
List / inspect endpoints	`list-endpoints` , `get-endpoint`
Endpoint health	`endpoint-health`
List workers (per-worker detail)	`list-endpoint-workers`
Read a worker's logs	`stream-worker-logs` ( `source=system` for the container lifecycle)
GPU stock per data center	`list-gpu-types` / `get-gpu-type` ( `include:["AVAILABILITY"]` , `product:["SERVERLESS"]` )
CUDA versions with capacity, per GPU type	the same read ( `cudaVersions` ); `get-capacity` for the matrix view
Update endpoint settings	`update-endpoint` (PATCH; send `gpu.pools` and `gpu.excludedTypes` together)
Pin GPUs / set GPU count	`set-endpoint-gpus`
Release history	`list-endpoint-releases`
Cancel one job	`cancel-job`
Purge the queue	`purge-endpoint-queue`
Submit / poll a job	`run-endpoint` , `get-job-status` , `runsync-endpoint`

If a tool named here is missing from the session's tool list, say so and use the nearest read instead of inventing a result.

能力	工具
列出/检查端点	`list-endpoints` , `get-endpoint`
端点健康状态	`endpoint-health`
列出worker（每个worker的详情）	`list-endpoint-workers`
读取worker日志	`stream-worker-logs` （容器生命周期请使用 `source=system` ）
各数据中心GPU库存	`list-gpu-types` / `get-gpu-type` （ `include:["AVAILABILITY"]` , `product:["SERVERLESS"]` ）
各GPU类型有可用容量的CUDA版本	同上读取操作（ `cudaVersions` ）；矩阵视图请使用 `get-capacity`
更新端点设置	`update-endpoint` （PATCH方式；请同时发送 `gpu.pools` 和 `gpu.excludedTypes` ）
固定GPU/设置GPU数量	`set-endpoint-gpus`
发布历史	`list-endpoint-releases`
取消单个作业	`cancel-job`
清空队列	`purge-endpoint-queue`
提交/轮询作业	`run-endpoint` , `get-job-status` , `runsync-endpoint`

如果会话的工具列表中缺少此处提及的某个工具，请说明情况，并使用最接近的读取工具代替，不要编造结果。

Contract

约定

The cross-journey answer contract (definite facts from reads, commit-don't-hedge, mutations bind to this conversation's creations, a standalone final message, honest failure, granted tools only) is defined in the

runpod-mcp

skill and applies to every reply from this journey. If the

runpod-mcp

skill has not been loaded in this conversation, load it now, before the next tool call: its rules on which resources you may change, when a request is the go, and when to stop for the user are not repeated here.

跨流程答复约定（基于读取的确定事实、明确承诺不模糊、修改仅绑定本会话创建的资源、独立的最终消息、诚实的失败反馈、仅使用已授权工具）已在

runpod-mcp

技能中定义，适用于本流程的每一次回复。如果本次会话中尚未加载

runpod-mcp

技能，请在下一次工具调用前加载：其中关于可修改哪些资源、何时可以执行请求、何时需要停下询问用户的规则，此处不再重复。