debug-inference

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Debug Inference

推理调试

Diagnose inference as ordinary provider-authorized network traffic. OpenShell no longer supplies a managed inference route, rewrites request shapes, or selects a model. The application calls the provider's native endpoint and owns its base URL, model, request format, and timeout.

Use installed

openshell --help

output as the authority for command syntax. Refer to the published provider management guide and provider profile guide for current behavior.

将推理故障作为普通的Provider授权网络流量问题进行诊断。OpenShell不再提供托管推理路由、重写请求结构或选择模型。应用直接调用Provider的原生端点，并自行管理其基础URL、模型、请求格式和超时设置。

请以已安装版本的

openshell --help

输出作为命令语法的权威依据。有关当前行为，请参阅已发布的Provider管理指南和Provider配置文件指南。

Diagnostic Workflow

诊断工作流

1. Confirm Gateway and Sandbox Context

1. 确认网关与沙箱上下文

bash

openshell status
openshell gateway info
openshell sandbox get <sandbox>

For a host-local model server,

host.openshell.internal

identifies the machine running the gateway. It does not identify the operator's laptop when the gateway is remote. A server listening only on

127.0.0.1

may also be unreachable from a container; bind it to an address reachable from the gateway runtime.

bash

openshell status
openshell gateway info
openshell sandbox get <sandbox>

对于本地主机部署的模型服务器，

host.openshell.internal

指向运行网关的机器。当网关为远程部署时，该地址不会指向操作员的笔记本电脑。仅监听

127.0.0.1

的服务器可能无法从容器访问；请将其绑定到网关运行时可到达的地址。

2. Inspect the Provider and Its Profile

2. 检查Provider及其配置文件

bash

openshell provider get <provider>
openshell profile export <profile-id> -o yaml

Check that the profile:

Names the exact endpoint host, port, and protocol the client calls.
Allows the client binary.
Declares the credential key and intended authentication style.
Uses narrow HTTP rules when the provider should expose only part of an API.

For a custom or self-hosted OpenAI-compatible endpoint, import an endpoint-bearing profile. A base URL stored only in provider configuration does not authorize a new endpoint.

bash

openshell profile lint -f ./provider-profile.yaml
openshell profile import -f ./provider-profile.yaml
openshell provider create --name <provider> --type <profile-id>

Add the required

--credential KEY

--credential KEY=VALUE

arguments shown by the profile. Never broaden endpoint policy merely to silence a credential binding error.

bash

openshell provider get <provider>
openshell profile export <profile-id> -o yaml

请检查配置文件是否满足以下要求：

准确指定客户端调用的端点主机、端口和协议。
允许客户端二进制文件访问。
声明凭证密钥和预期的认证方式。
当Provider仅应暴露部分API时，使用严格的HTTP规则。

对于自定义或自托管的OpenAI兼容端点，请导入包含端点配置的配置文件。仅存储在Provider配置中的基础URL不会为新端点提供授权。

bash

openshell profile lint -f ./provider-profile.yaml
openshell profile import -f ./provider-profile.yaml
openshell provider create --name <provider> --type <profile-id>

添加配置文件中显示的必填

--credential KEY

或

--credential KEY=VALUE

参数。切勿为了消除凭证绑定错误而放宽端点策略。

3. Confirm Attachment

3. 确认挂载状态

bash

openshell sandbox provider list <sandbox>
openshell sandbox provider attach <sandbox> <provider> --wait --timeout 30

Save the change's

receipt_id

and use

openshell sandbox provider status <sandbox> <provider> --receipt <receipt-id> --wait --timeout 30

to check when it takes effect. Success confirms that the sandbox applied the credentials, policy, and environment for new processes. If the result is pending, failed, withheld, or superseded, inspect its reason before launching the client.

Launch the client after the attachment wait succeeds so it receives the updated environment:

bash

openshell sandbox exec <sandbox> -- <client-command>

After updating an ordinary static provider, wait for that change and launch a new client. An existing process keeps its revision-scoped reference; readiness does not make the old reference resolve the replacement value. Diagnose managed-refresh credentials according to their own lifecycle.

Keep credentials and issued references out of diagnostic output. Acknowledged detach revokes future credential resolution and removes the reference from future process environments. Requests already forwarded may still finish:

bash

openshell sandbox provider detach <sandbox> <provider> --wait --timeout 30

bash

openshell sandbox provider list <sandbox>
openshell sandbox provider attach <sandbox> <provider> --wait --timeout 30

保存更改的

receipt_id

，并使用

openshell sandbox provider status <sandbox> <provider> --receipt <receipt-id> --wait --timeout 30

命令检查其何时生效。成功状态表示沙箱已为新进程应用了凭证、策略和环境变量。如果结果为待处理、失败、已暂缓或已取代，请在启动客户端前检查原因。

挂载等待成功后再启动客户端，以确保客户端获取更新后的环境：

bash

openshell sandbox exec <sandbox> -- <client-command>

更新普通静态Provider后，请等待更改生效并启动新的客户端。现有进程会保留其对应修订版本的引用；就绪状态不会使旧引用解析为新的替换值。请根据托管刷新凭证自身的生命周期对其进行诊断。

请勿在诊断输出中包含凭证和已颁发的引用信息。已确认的卸载操作会撤销后续的凭证解析，并从未来的进程环境中移除该引用。已转发的请求可能仍会继续执行完成：

bash

openshell sandbox provider detach <sandbox> <provider> --wait --timeout 30

4. Verify Native Client Configuration

4. 验证原生客户端配置

The application must use the real upstream contract:

Native provider base URL, not the retired managed virtual endpoint.
Real model ID, not a placeholder that OpenShell used to rewrite.
Native OpenAI, Anthropic, Vertex, or other provider request shape.
Application-owned timeout and retry settings.
The credential environment variable declared by the attached profile.

Probe the exact endpoint from a newly launched sandbox process. Start with a non-secret discovery endpoint when the provider offers one, then send a minimal inference request using the provider's documented API shape.

应用必须遵循真实的上游接口约定：

Provider原生基础URL，而非已停用的托管虚拟端点。
真实的模型ID，而非OpenShell过去用于重写的占位符。
原生的OpenAI、Anthropic、Vertex或其他Provider的请求结构。
应用自行管理的超时和重试设置。
已挂载配置文件中声明的凭证环境变量。

请从新启动的沙箱进程中探测目标端点的连通性。如果Provider提供非涉密的发现端点，可先使用该端点进行探测，再按照Provider官方文档规定的API结构发送最小化推理请求。

5. Interpret Common Failures

5. 常见故障解读

Symptom	Likely cause	Fix
`credential_placeholder_in_request_body`	A body reference is invalid/revoked, or classification metadata is unavailable	Check the controlled denial reason; remove the reference from conversation history or restore provider access. Do not enable body credential rewriting or bypass flags to send tool output. Unknown literals and valid issued placeholders pass unchanged, including the model provider’s own placeholder. Header resolution does not enable body rewriting.
A retired managed endpoint fails DNS resolution	Client still uses the removed managed endpoint	Configure the provider's native base URL and attach an endpoint-bearing provider profile
Direct request is denied	Missing attachment, endpoint policy, HTTP rule, or binary authorization	Inspect the attached provider profile and sandbox effective policy
`credential_endpoint_mismatch`	Credential profile does not authorize the request recipient	Correct the host/port/path or import a narrowly scoped profile for the intended endpoint
`request_authority_mismatch`	HTTP authority differs from the CONNECT destination	Use the same host and effective port in both authorities
Credential variable is absent	Provider was not attached when this process launched, or profiles collide on a key	Attach the provider and launch a new process; resolve duplicate keys explicitly
Upstream rejects the model or body	Client relied on removed model/request rewriting	Configure the real model and provider-native request format in the application
`127.0.0.1` works on the host but not in the sandbox	Loopback refers to different runtime	Use `host.openshell.internal` or another gateway-reachable endpoint and profile
Host-local request times out	Server bind address, gateway topology, or host firewall blocks container-to-host traffic	Verify the listener and permit only the required gateway network path and port

症状	可能原因	解决方法
`credential_placeholder_in_request_body`	请求体中的引用无效/已被撤销，或分类元数据不可用	检查受控拒绝的原因；从对话历史中移除该引用，或恢复Provider的访问权限。请勿启用请求体凭证重写或绕过标志来发送工具输出。未知字面量和有效的已颁发占位符会原样传递，包括模型提供商自身的占位符。请求头解析不会启用请求体重写功能。
已停用的托管端点DNS解析失败	客户端仍在使用已下线的托管端点	配置Provider的原生基础URL，并挂载包含端点配置的Provider配置文件
直接请求被拒绝	缺少挂载关系、端点策略、HTTP规则或二进制文件授权	检查已挂载的Provider配置文件和沙箱的生效策略
`credential_endpoint_mismatch`	凭证配置文件未对请求目标授予权限	修正主机/端口/路径，或为目标端点导入权限范围更严格的配置文件
`request_authority_mismatch`	HTTP authority与CONNECT目标不一致	请确保两个authority使用相同的主机和生效端口
凭证变量不存在	进程启动时未挂载Provider，或多个配置文件的密钥存在冲突	挂载Provider并启动新进程；明确解决重复密钥的问题
上游服务拒绝了模型或请求体	客户端依赖已下线的模型/请求重写功能	在应用中配置真实的模型和Provider原生的请求格式
`127.0.0.1` 在主机上可用但在沙箱中不可用	环回地址指向不同的运行时环境	使用 `host.openshell.internal` 或其他网关可访问的端点及对应配置文件
本地主机请求超时	服务器绑定地址、网关拓扑或主机防火墙阻止了容器到主机的流量	验证监听器配置，仅允许所需的网关网络路径和端口访问

Host-Local Inference Checklist

本地主机推理检查清单

For Ollama, LM Studio, vLLM, SGLang, TRT-LLM, and local NIM deployments:

Verify the engine from the gateway host.
Verify it listens on an address reachable from the gateway runtime.
Import a custom profile naming
```
host.openshell.internal
```
and the actual port.
Restrict the profile to the intended binaries and API paths.
Create and attach the provider.
Configure the application's base URL, model, and timeout.
Probe the native endpoint from a newly launched sandbox process.

适用于Ollama、LM Studio、vLLM、SGLang、TRT-LLM和本地NIM部署场景：

从网关主机验证引擎运行正常。
验证引擎监听的地址可被网关运行时访问。
导入指定
```
host.openshell.internal
```
和实际端口的自定义配置文件。
将配置文件的访问权限限制为预期的二进制文件和API路径。
创建并挂载Provider。
配置应用的基础URL、模型和超时时间。
从新启动的沙箱进程中探测原生端点的连通性。

Reporting

问题上报

Report:

The active gateway and whether topology contributes to the failure.
The provider, profile, attachment, endpoint, and client binary involved.
The exact failed host, port, path, and request authority without secrets.
Whether the client still relies on removed managed-routing behavior.
The narrowest profile, attachment, or application configuration change that resolves the problem.

上报时请提供以下信息：

当前活跃的网关，以及拓扑结构是否为故障诱因。
涉及的Provider、配置文件、挂载关系、端点和客户端二进制文件。
准确的故障主机、端口、路径和请求authority（不含涉密信息）。
客户端是否仍依赖已下线的托管路由功能。
可解决问题的最小范围配置调整（配置文件、挂载或应用配置）。