huawei-cloud-ascend-models-deploy

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Huawei Cloud Ascend Models Deploy

华为云Ascend模型部署

Deploy and test large language models on Huawei Cloud Ascend DevServer (910B series). Supports single-machine and dual-machine deployment, model inference testing, and deployment monitoring.

在华为云Ascend DevServer（910B系列）上部署并测试大语言模型，支持单机与双机部署、模型推理测试及部署监控。

Overview

概述

This skill deploys and tests large language models on Huawei Cloud Ascend DevServer (910B series). Supports single-machine and dual-machine deployment for LLM, VL, Embedding, and Rerank models.

Related Skills (Agent orchestrated, no direct call, Rule 3):

```
huawei-cloud-ascend-remote-connect
```
- SSH connection to DevServer (prerequisite for deployment)
```
huawei-cloud-ascend-command
```
- NPU status check and monitoring (prerequisite and post-deploy monitoring)

Capabilities:

Model deployment (single-node, dual-node)
Inference testing (LLM chat, VL multimodal, Embedding, Rerank)
Deployment log and status monitoring
Model catalog and script auto-matching

Deployment Workflow (Agent orchestrated):

Agent calls
```
huawei-cloud-ascend-remote-connect
```
to establish SSH connection
Agent calls
```
huawei-cloud-ascend-command
```
to check NPU health and availability
Agent calls this skill (
```
huawei-cloud-ascend-models-deploy
```
) to deploy model
Agent calls
```
huawei-cloud-ascend-command
```
to monitor NPU status during deployment

本技能用于在华为云Ascend DevServer（910B系列）上部署并测试大语言模型，支持LLM、VL、Embedding和Rerank模型的单机与双机部署。

相关技能（由Agent编排，不可直接调用，规则3）：

```
huawei-cloud-ascend-remote-connect
```
- 与DevServer建立SSH连接（部署前提）
```
huawei-cloud-ascend-command
```
- NPU状态检查与监控（部署前提及部署后监控）

功能:

模型部署（单机、双机）
推理测试（LLM对话、VL多模态、Embedding、Rerank）
部署日志与状态监控
模型目录与脚本自动匹配

部署流程（由Agent编排）:

Agent调用
```
huawei-cloud-ascend-remote-connect
```
建立SSH连接
Agent调用
```
huawei-cloud-ascend-command
```
检查NPU健康状态与可用性
Agent调用本技能（
```
huawei-cloud-ascend-models-deploy
```
）部署模型
Agent调用
```
huawei-cloud-ascend-command
```
在部署过程中监控NPU状态

Architecture

架构

System Architecture Diagram

系统架构图

┌─────────────────────────────────────────────────────────────────────┐
│                         Agent Orchestration                         │
│  ┌─────────────────────────────────────────────────────────────┐    │
│  │  1. SSH connect (remote-connect)                             │    │
│  │  2. NPU health check (ascend-command)                        │    │
│  │  3. Deploy model (this skill)                                 │    │
│  │  4. Monitor NPU (ascend-command)                             │    │
│  └────────────────────────────┬────────────────────────────────┘    │
│                               │ Explicit param passing (Rule 1)    │
│                               ▼                                     │
├─────────────────────────────────────────────────────────────────────┤
│              Huawei Cloud Ascend Models Deploy                      │
│                      (Stateless, Rule 2)                            │
├─────────────────────────────────────────────────────────────────────┤
│  ┌──────────────────┐    ┌──────────────────────────────────┐      │
│  │  Natural Language│    │          Deploy Helper           │      │
│  │     Commands     │───▶│  - Model Matching & Catalog      │      │
│  └──────────────────┘    │  - Script Auto-Match             │      │
│                          │  - Command Generation            │      │
│                          └──────────────────────────────────┘      │
│                                           │                         │
│          ┌─────────────────────────────────┼──────────────┐        │
│          ▼                                 ▼              ▼        │
│  ┌───────────────┐              ┌─────────────────┐ ┌────────┐    │
│  │ Model         │              │ Inference       │ │ Log    │    │
│  │ Deployment    │              │ Testing         │ │ Status │    │
│  │               │              │                 │ │        │    │
│  │ • Single-node │              │ • LLM Chat      │ │ • View │    │
│  │ • Dual-node   │              │ • VL Multimodal │ │ • Check│    │
│  │ • 910B Series │              │ • Embedding     │ │        │    │
│  └───────────────┘              │ • Rerank        │ └────────┘    │
│                                 └─────────────────┘               │
└─────────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────────┐
│                         Agent Orchestration                         │
│  ┌─────────────────────────────────────────────────────────────┐    │
│  │  1. SSH connect (remote-connect)                             │    │
│  │  2. NPU health check (ascend-command)                        │    │
│  │  3. Deploy model (this skill)                                 │    │
│  │  4. Monitor NPU (ascend-command)                             │    │
│  └────────────────────────────┬────────────────────────────────┘    │
│                               │ Explicit param passing (Rule 1)    │
│                               ▼                                     │
├─────────────────────────────────────────────────────────────────────┤
│              Huawei Cloud Ascend Models Deploy                      │
│                      (Stateless, Rule 2)                            │
├─────────────────────────────────────────────────────────────────────┤
│  ┌──────────────────┐    ┌──────────────────────────────────┐      │
│  │  Natural Language│    │          Deploy Helper           │      │
│  │     Commands     │───▶│  - Model Matching & Catalog      │      │
│  └──────────────────┘    │  - Script Auto-Match             │      │
│                          │  - Command Generation            │      │
│                          └──────────────────────────────────┘      │
│                                           │                         │
│          ┌─────────────────────────────────┼──────────────┐        │
│          ▼                                 ▼              ▼        │
│  ┌───────────────┐              ┌─────────────────┐ ┌────────┐    │
│  │ Model         │              │ Inference       │ │ Log    │    │
│  │ Deployment    │              │ Testing         │ │ Status │    │
│  │               │              │                 │ │        │    │
│  │ • Single-node │              │ • LLM Chat      │ │ • View │    │
│  │ • Dual-node   │              │ • VL Multimodal │ │ • Check│    │
│  │ • 910B Series │              │ • Embedding     │ │        │    │
│  └───────────────┘              │ • Rerank        │ └────────┘    │
│                                 └─────────────────┘               │
└─────────────────────────────────────────────────────────────────────┘

Agent Orchestration Flow

Agent编排流程

User request: "Deploy Qwen2.5-72B on DevServer 116.204.23.145"
         │
         ▼
┌─────────────────────────────────────────────────────────────┐
│ Agent Step 1: SSH Connection                                 │
│   → Call huawei-cloud-ascend-remote-connect                  │
│   → Pass: host, user, password (explicit, Rule 1)            │
└─────────────────────────────────────────────────────────────┘
         │
         ▼
┌─────────────────────────────────────────────────────────────┐
│ Agent Step 2: NPU Health Check                               │
│   → Call huawei-cloud-ascend-command                         │
│   → Check: NPU list, health, HBM availability                │
│   → Fail if NPU not healthy or insufficient HBM              │
└─────────────────────────────────────────────────────────────┘
         │
         ▼
┌─────────────────────────────────────────────────────────────┐
│ Agent Step 3: Deploy Model (this skill)                      │
│   → Match model from catalog                                 │
│   → Generate deploy script                                   │
│   → Execute deployment                                        │
│   → Stateless execution (Rule 2)                             │
└─────────────────────────────────────────────────────────────┘
         │
         ▼
┌─────────────────────────────────────────────────────────────┐
│ Agent Step 4: Monitor NPU                                    │
│   → Call huawei-cloud-ascend-command                         │
│   → Monitor: HBM usage, temperature, processes               │
└─────────────────────────────────────────────────────────────┘
         │
         ▼
      Deployment Complete

User request: "Deploy Qwen2.5-72B on DevServer 116.204.23.145"
         │
         ▼
┌─────────────────────────────────────────────────────────────┐
│ Agent Step 1: SSH Connection                                 │
│   → Call huawei-cloud-ascend-remote-connect                  │
│   → Pass: host, user, password (explicit, Rule 1)            │
└─────────────────────────────────────────────────────────────┘
         │
         ▼
┌─────────────────────────────────────────────────────────────┐
│ Agent Step 2: NPU Health Check                               │
│   → Call huawei-cloud-ascend-command                         │
│   → Check: NPU list, health, HBM availability                │
│   → Fail if NPU not healthy or insufficient HBM              │
└─────────────────────────────────────────────────────────────┘
         │
         ▼
┌─────────────────────────────────────────────────────────────┐
│ Agent Step 3: Deploy Model (this skill)                      │
│   → Match model from catalog                                 │
│   → Generate deploy script                                   │
│   → Execute deployment                                        │
│   → Stateless execution (Rule 2)                             │
└─────────────────────────────────────────────────────────────┘
         │
         ▼
┌─────────────────────────────────────────────────────────────┐
│ Agent Step 4: Monitor NPU                                    │
│   → Call huawei-cloud-ascend-command                         │
│   → Monitor: HBM usage, temperature, processes               │
└─────────────────────────────────────────────────────────────┘
         │
         ▼
      Deployment Complete

Related Skills Table

Skill	Purpose	Orchestration Stage
`huawei-cloud-ascend-remote-connect`	SSH connection	Pre-deploy: Establish connection to DevServer
`huawei-cloud-ascend-command`	NPU management	Pre-deploy: Health check; Post-deploy: Monitoring

Skill	用途	编排阶段
`huawei-cloud-ascend-remote-connect`	SSH连接	部署前：建立与DevServer的连接
`huawei-cloud-ascend-command`	NPU管理	部署前：健康检查；部署后：监控

Prerequisites

前提条件

Prerequisite check: Ascend 910B series required
Supported: 910B1, 910B2, 910B3, 910B4

Unsupported: 910A, 310, 310P, etc.
Check with:
npu-smi info

前提检查：必须为Ascend 910B系列
支持机型：910B1、910B2、910B3、910B4

不支持机型：910A、310、310P等
检查命令：
npu-smi info

Mandatory Rules (AI Must Follow)

强制规则（AI必须遵守）

Never guess commands from memory — Must read "Deploy Script Auto-Match" section
Must call deploy_helper.py first — Confirm model category and script URL
Different models use different scripts:
- LLM / Embedding / Rerank →
```
deploy-large-models.sh
```
- VL →
```
deploy-qwen3-vl-model.sh
```
- OpenSource →
```
deploy-ai-models.sh
```
Must validate before deployment — Port, NPU, model, card count
Show command and wait for confirmation — Sensitive operation, never execute directly

禁止凭记忆猜测命令 — 必须查看「部署脚本自动匹配」章节
必须先调用deploy_helper.py — 确认模型类别与脚本URL
不同模型使用不同脚本:
- LLM / Embedding / Rerank →
```
deploy-large-models.sh
```
- VL →
```
deploy-qwen3-vl-model.sh
```
- OpenSource →
```
deploy-ai-models.sh
```
部署前必须验证 — 端口、NPU、模型、卡数
展示命令并等待确认 — 敏感操作，禁止直接执行

Natural Language Understanding Rules

自然语言理解规则

Extract key information from user natural language and assemble commands accurately.

从用户自然语言中提取关键信息，准确组装命令。

Operation Type Detection

操作类型检测

Keywords	Operation
deploy / start / launch	Single-machine deployment
dual-machine / two-node / dual-node	Dual-machine deployment
test / inference / call	Test (execute)
write command / generate command	Write test command (generate only, no execute)
deployment log / view log	View deployment log
deployment status / is ready	View deployment status
model list / supported models	Show model catalog
parameter help / API parameters	Show parameter manual

关键词	操作
deploy / start / launch	单机部署
dual-machine / two-node / dual-node	双机部署
test / inference / call	测试（执行）
write command / generate command	生成测试命令（仅生成，不执行）
deployment log / view log	查看部署日志
deployment status / is ready	查看部署状态
model list / supported models	展示模型目录
parameter help / API parameters	展示参数手册

Information Extraction Rules

信息提取规则

Model Name (fuzzy match, case-insensitive, supports card count filter):

"qwen3-14b" → Qwen3-14B
"qwen3-235b" → Multiple matches, prefer Instruct version (Qwen3-235B-A22B-Instruct-2507), or ask user
"vl-32b" → Qwen3-VL-32B-Instruct
"bge-m3" → bge-m3
"qwen3-vl" + 2 cards → Match VL models with ≤2 cards, list for user to choose
"qwen3" + 2 cards → Match all Qwen3 models with ≤2 cards, list for user to choose
Multiple candidates → List all candidates (with card count and category), let user confirm
No match → Show full model catalog for user to select

Card Count:

"2 cards" / "use 2 cards" / "2 npus" → 2
"16 cards" / "16 npus" → 16
"dual-machine" → 16
Not specified → Use minimum card count from model catalog

Port:

"port 8022" / "port:8022" → 8022
Not specified → Default 8080

Missing Parameters (check each, prompt what is missing):

Missing model name → "Please specify model name" + show model list
Missing card count → "Please specify card count, e.g.: 2 cards" + show minimum cards for this model
Missing port → "Please specify port (default 8080), e.g.: port 8001"
Dual-machine missing head IP → "Please specify head node IP, e.g.: head:192.168.1.1"
Dual-machine missing worker IP → "Please specify worker node IP, e.g.: worker:192.168.1.2"

Head/Worker IP (dual-machine deployment):

"head:1.1.1.1" / "head node 1.1.1.1" → Head node IP
"worker:2.2.2.2" / "worker node 2.2.2.2" → Worker node IP

Prompt:

"prompt:hello" / "ask:hello" → Prompt text
Not specified → LLM default "hello", VL default "describe the image", Embedding default "I love shanghai", Rerank default "What is the capital of France?"

Image URL (VL test):

"image:https://xxx.jpg" / direct URL → Image URL
User sends image attachment → Auto-convert to base64 data URL
Not specified and testing multimodal model → Prompt user for image URL

Multimodal Capability Auto-Detection:

VL category → Supports multimodal
OpenSource: Qwen3.6-35B-A3B, Qwen3.6-27B → Supports multimodal
LLM category → Text only
Embedding → Text only
Rerank → Text only

Image URL Conversion (local image → data URL):

bash

undefined

模型名称（模糊匹配，大小写不敏感，支持卡数过滤）:

"qwen3-14b" → Qwen3-14B
"qwen3-235b" → 多个匹配结果，优先选择Instruct版本（Qwen3-235B-A22B-Instruct-2507），或询问用户
"vl-32b" → Qwen3-VL-32B-Instruct
"bge-m3" → bge-m3
"qwen3-vl" + 2卡 → 匹配支持≤2卡的VL模型，列出供用户选择
"qwen3" + 2卡 → 匹配支持≤2卡的所有Qwen3模型，列出供用户选择
多个候选结果 → 列出所有候选（含卡数与类别），让用户确认
无匹配结果 → 展示完整模型目录供用户选择

卡数:

"2 cards" / "use 2 cards" / "2 npus" → 2
"16 cards" / "16 npus" → 16
"dual-machine" → 16
未指定 → 使用模型目录中的最小卡数

端口:

"port 8022" / "port:8022" → 8022
未指定 → 默认8080

缺失参数（逐一检查，提示缺失内容）:

缺失模型名称 → "请指定模型名称" + 展示模型列表
缺失卡数 → "请指定卡数，例如：2 cards" + 展示该模型的最小卡数
缺失端口 → "请指定端口（默认8080），例如：port 8001"
双机部署缺失主节点IP → "请指定主节点IP，例如：head:192.168.1.1"
双机部署缺失工作节点IP → "请指定工作节点IP，例如：worker:192.168.1.2"

主/工作节点IP（双机部署）:

"head:1.1.1.1" / "head node 1.1.1.1" → 主节点IP
"worker:2.2.2.2" / "worker node 2.2.2.2" → 工作节点IP

提示词:

"prompt:hello" / "ask:hello" → 提示文本
未指定 → LLM默认"hello"，VL默认"describe the image"，Embedding默认"I love shanghai"，Rerank默认"What is the capital of France?"

图片URL（VL测试）:

"image:https://xxx.jpg" / 直接URL → 图片URL
用户发送图片附件 → 自动转换为base64数据URL
未指定且测试多模态模型 → 提示用户提供图片URL

多模态能力自动检测:

VL类别 → 支持多模态
OpenSource: Qwen3.6-35B-A3B、Qwen3.6-27B → 支持多模态
LLM类别 → 仅支持文本
Embedding → 仅支持文本
Rerank → 仅支持文本

图片URL转换（本地图片→数据URL）:

bash

undefined

Efficient base64 conversion

IMG_B64=$(base64 -w 0 ${local_image_path}) IMG_URL="data:image/jpeg;base64,${IMG_B64}"


**Advanced Parameters (optional):**
- "max_tokens:64" → max_tokens=64
- "temperature:0.7" → temperature=0.7
- "stream" → stream=true
- "system:You are assistant" → system_prompt
- "disable thinking" / "no thinking" → chat_template_kwargs: {"enable_thinking": false}
- (Default = thinking mode enabled)

**Thinking Mode:**
Qwen3/Qwen3.6 models default to thinking mode, outputting reasoning process before final response.
- Enable thinking: Higher quality, more token consumption
- Disable thinking: Direct output, less token consumption, suitable for simple queries
- Request-level control via `"chat_template_kwargs": {"enable_thinking": false/true}`

---

IMG_B64=$(base64 -w 0 ${local_image_path}) IMG_URL="data:image/jpeg;base64,${IMG_B64}"


**高级参数（可选）**:
- "max_tokens:64" → max_tokens=64
- "temperature:0.7" → temperature=0.7
- "stream" → stream=true
- "system:You are assistant" → system_prompt
- "disable thinking" / "no thinking" → chat_template_kwargs: {"enable_thinking": false}
- （默认 = 启用思考模式）

**思考模式**:
Qwen3/Qwen3.6模型默认启用思考模式，在最终响应前输出推理过程。
- 启用思考：质量更高，token消耗更多
- 禁用思考：直接输出，token消耗更少，适用于简单查询
- 通过`"chat_template_kwargs": {"enable_thinking": false/true}`在请求级别控制

---

Supported Machine Types

支持的机型

Only Ascend 910B series (910B1 / 910B2 / 910B3 / 910B4). Must check NPU model before deployment, reject non-910B series.

仅支持Ascend 910B系列（910B1 / 910B2 / 910B3 / 910B4）。部署前必须检查NPU型号，拒绝非910B系列机型。

Model Catalog

模型目录

Large Language Models (LLM) — Endpoint: /v1/chat/completions

大语言模型（LLM） — 端点：/v1/chat/completions

Model	Min Cards
Qwen3-14B	1
Qwen3-30B-A3B-Instruct-2507	2
Qwen3-32B	2
Qwen3-235B-A22B-Thinking-2507	16
Qwen3-235B-A22B-Instruct-2507	16
DeepSeek-R1-Distill-Llama-70B	4

Model	最小卡数
Qwen3-14B	1
Qwen3-30B-A3B-Instruct-2507	2
Qwen3-32B	2
Qwen3-235B-A22B-Thinking-2507	16
Qwen3-235B-A22B-Instruct-2507	16
DeepSeek-R1-Distill-Llama-70B	4

Vision-Language (VL) — Endpoint: /v1/chat/completions

视觉语言模型（VL） — 端点：/v1/chat/completions

Model	Min Cards
Qwen3-VL-30B-A3B-Instruct	2
Qwen3-VL-32B-Instruct	2
Qwen3-VL-235B-A22B-Instruct	16
Qwen3-VL-235B-A22B-Instruct-W8A8	8

Model	最小卡数
Qwen3-VL-30B-A3B-Instruct	2
Qwen3-VL-32B-Instruct	2
Qwen3-VL-235B-A22B-Instruct	16
Qwen3-VL-235B-A22B-Instruct-W8A8	8

Embedding — Endpoint: /v1/embeddings (V0 backend only, single card only)

Embedding模型 — 端点：/v1/embeddings（仅V0后端，仅支持单卡）

Model	Min Cards	Multi-card
Qwen3-Embedding-8B	1	No
bge-large-zh-v1.5	1	No
bge-m3	1	No

Model	最小卡数	多卡支持
Qwen3-Embedding-8B	1	否
bge-large-zh-v1.5	1	否
bge-m3	1	否

Rerank — Endpoint: /v1/rerank (single card only)

Rerank模型 — 端点：/v1/rerank（仅支持单卡）

Model	Min Cards	Multi-card
Qwen3-Reranker-8B	1	No
bge-reranker-v2-m3	1	No

Model	最小卡数	多卡支持
Qwen3-Reranker-8B	1	否
bge-reranker-v2-m3	1	否

OpenSource (Multimodal)

开源模型（多模态）

Model	Min Cards	Capability
Qwen3.6-35B-A3B	2	Text + Image (MoE)
Qwen3.6-27B	2	Text + Image (MoE)
Qwen3-Next-80B-A3B-Instruct	4	Large language model
DeepSeek-V4-Flash-w8a8-mtp	8	Large language model

Model	最小卡数	能力
Qwen3.6-35B-A3B	2	文本+图像（MoE）
Qwen3.6-27B	2	文本+图像（MoE）
Qwen3-Next-80B-A3B-Instruct	4	大语言模型
DeepSeek-V4-Flash-w8a8-mtp	8	大语言模型

Deploy Script Auto-Match (Must use, never guess script URL)

部署脚本自动匹配（必须使用，禁止猜测脚本URL）

Script Path:

scripts/deploy_helper.py

Match Rules (hardcoded, 100% accurate):

Model Category	Deploy Script	Notes
LLM	`deploy-large-models.sh`	Shared with Embedding/Rerank
Embedding	`deploy-large-models.sh`	Same as above
Rerank	`deploy-large-models.sh`	Same as above
VL	`deploy-qwen3-vl-model.sh`	Multimodal specific
OpenSource	`deploy-ai-models.sh`	OpenSource specific

Usage:

bash

undefined

脚本路径:

scripts/deploy_helper.py

匹配规则（硬编码，100%准确）:

模型类别	部署脚本	说明
LLM	`deploy-large-models.sh`	与Embedding/Rerank共用
Embedding	`deploy-large-models.sh`	同上
Rerank	`deploy-large-models.sh`	同上
VL	`deploy-qwen3-vl-model.sh`	多模态专用
OpenSource	`deploy-ai-models.sh`	开源模型专用

用法:

bash

undefined

Match model (returns category, script URL, min cards, etc.)

匹配模型（返回类别、脚本URL、最小卡数等）

python3 scripts/deploy_helper.py match <model_name>

Generate deploy command directly

直接生成部署命令

python3 scripts/deploy_helper.py command <model_name> <cards> <port>

List all models (optional category filter)

列出所有模型（可选类别过滤）

python3 scripts/deploy_helper.py list [LLM|VL|Embedding|Rerank|OpenSource]


**AI must call `deploy_helper.py match` first to confirm category and script, then use returned `deploy_url` to assemble command. Never guess from memory!**

---

python3 scripts/deploy_helper.py list [LLM|VL|Embedding|Rerank|OpenSource]


**AI必须先调用`deploy_helper.py match`确认类别与脚本，再使用返回的`deploy_url`组装命令，禁止凭记忆猜测！**

---

Core Commands

核心命令

Core commands for model deployment and testing. See Operation Flow for detailed steps.

Command	Description
`deploy <model> <port>`	Deploy model on single machine
`deploy <model> <port> <cards>`	Deploy with specified card count
`dual-machine deploy <model> head:<IP> worker:<IP> port:<PORT>`	Deploy on dual-machine cluster
`test <model> <port>`	Test model inference
`deployment log`	View deployment log
`deployment status`	Check deployment status
`model list`	Show supported models

模型部署与测试的核心命令，详细步骤请查看「操作流程」章节。

命令	描述
`deploy <model> <port>`	单机部署模型
`deploy <model> <port> <cards>`	指定卡数部署模型
`dual-machine deploy <model> head:<IP> worker:<IP> port:<PORT>`	在双机集群部署模型
`test <model> <port>`	测试模型推理
`deployment log`	查看部署日志
`deployment status`	检查部署状态
`model list`	展示支持的模型

Operation Flow

操作流程

I. Deployment

I. 部署

1. Pre-deployment Check (Must execute every time, cannot skip)

1. 部署前检查（每次必须执行，不可跳过）

Check in order, stop if any fails:

NPU Model Check — Agent calls
```
huawei-cloud-ascend-command
```
to check chip model, reject non-910B series
NPU Card Count Check — Agent calls
```
huawei-cloud-ascend-command
```
to check available cards, confirm >= required cards
User Card Count Check — User-specified cards must be >= minimum and within supported range (1,2,4,8,16)
Embedding/Rerank Single Card Check — Embedding and Rerank only support single card, reject multi-card
Port Occupancy Check — Agent calls
```
huawei-cloud-ascend-remote-connect
```
to run
```
ss -tlnp | grep :port
```
, notify if occupied
SSH Connectivity Check — For dual-machine, verify both head and worker nodes are SSH accessible

按顺序检查，任意一项失败则停止：

NPU型号检查 — Agent调用
```
huawei-cloud-ascend-command
```
检查芯片型号，拒绝非910B系列
NPU卡数检查 — Agent调用
```
huawei-cloud-ascend-command
```
检查可用卡数，确认≥所需卡数
用户指定卡数检查 — 用户指定的卡数必须≥最小卡数且在支持范围内（1、2、4、8、16）
Embedding/Rerank单卡检查 — Embedding和Rerank仅支持单卡，拒绝多卡部署
端口占用检查 — Agent调用
```
huawei-cloud-ascend-remote-connect
```
执行
```
ss -tlnp | grep :port
```
，若占用则通知用户
SSH连通性检查 — 双机部署时，验证主节点与工作节点均能通过SSH访问

2. Single-machine Deployment

2. 单机部署

User says: "deploy model_name port XXXX" or "deploy model_name port XXXX N cards"

Before deploying, must SSH execute
mkdir -p /home/modelarts-agent
to ensure directory exists.

LLM / Embedding / Rerank Command Template:

bash

nohup bash -c 'export model_name=${model} && export required_cards=${cards} && export port=${port} && wget -P /home/modelarts-agent/ https://documentation-samples-17.obs.cn-north-9.myhuaweicloud.com/solution-as-code-publicbucket/solution-as-code-module/quickly-deploy-llm-on-modelarts-lite-devserver/userdata/deploy-large-models/single-machine/deploy-large-models.sh && chmod 755 /home/modelarts-agent/deploy-large-models.sh && sh /home/modelarts-agent/deploy-large-models.sh ${model} ${cards} ${port}' > /home/modelarts-agent/deploy_${model}.log 2>&1 &

VL Multimodal Command Template:

bash

nohup bash -c 'export model_name=${model} && export required_cards=${cards} && export port=${port} && wget -P /home/modelarts-agent/ https://documentation-samples-17.obs.cn-north-9.myhuaweicloud.com/solution-as-code-publicbucket/solution-as-code-module/quickly-deploy-llm-on-modelarts-lite-devserver/userdata/deploy-vl-model/single-machine/deploy-qwen3-vl-model.sh && chmod 755 /home/modelarts-agent/deploy-qwen3-vl-model.sh && sh /home/modelarts-agent/deploy-qwen3-vl-model.sh ${model} ${cards} ${port}' > /home/modelarts-agent/deploy_${model}.log 2>&1 &

OpenSource Command Template:

bash

nohup bash -c 'export model_name=${model} && export required_cards=${cards} && export port=${port} && wget -P /home/modelarts-agent/ https://documentation-samples-17.obs.cn-north-9.myhuaweicloud.com/solution-as-code-publicbucket/solution-as-code-module/quickly-deploy-llm-on-modelarts-lite-devserver/userdata/deploy-large-models/single-machine/open_source/deploy-ai-models.sh && chmod 755 /home/modelarts-agent/deploy-ai-models.sh && sh /home/modelarts-agent/deploy-ai-models.sh ${model} ${cards} ${port}' > /home/modelarts-agent/deploy_${model}.log 2>&1 &

用户指令："deploy model_name port XXXX" 或 "deploy model_name port XXXX N cards"

部署前，必须通过SSH执行
mkdir -p /home/modelarts-agent
确保目录存在。

LLM / Embedding / Rerank命令模板:

bash

nohup bash -c 'export model_name=${model} && export required_cards=${cards} && export port=${port} && wget -P /home/modelarts-agent/ https://documentation-samples-17.obs.cn-north-9.myhuaweicloud.com/solution-as-code-publicbucket/solution-as-code-module/quickly-deploy-llm-on-modelarts-lite-devserver/userdata/deploy-large-models/single-machine/deploy-large-models.sh && chmod 755 /home/modelarts-agent/deploy-large-models.sh && sh /home/modelarts-agent/deploy-large-models.sh ${model} ${cards} ${port}' > /home/modelarts-agent/deploy_${model}.log 2>&1 &

VL多模态命令模板:

bash

nohup bash -c 'export model_name=${model} && export required_cards=${cards} && export port=${port} && wget -P /home/modelarts-agent/ https://documentation-samples-17.obs.cn-north-9.myhuaweicloud.com/solution-as-code-publicbucket/solution-as-code-module/quickly-deploy-llm-on-modelarts-lite-devserver/userdata/deploy-vl-model/single-machine/deploy-qwen3-vl-model.sh && chmod 755 /home/modelarts-agent/deploy-qwen3-vl-model.sh && sh /home/modelarts-agent/deploy-qwen3-vl-model.sh ${model} ${cards} ${port}' > /home/modelarts-agent/deploy_${model}.log 2>&1 &

开源模型命令模板:

bash

nohup bash -c 'export model_name=${model} && export required_cards=${cards} && export port=${port} && wget -P /home/modelarts-agent/ https://documentation-samples-17.obs.cn-north-9.myhuaweicloud.com/solution-as-code-publicbucket/solution-as-code-module/quickly-deploy-llm-on-modelarts-lite-devserver/userdata/deploy-large-models/single-machine/open_source/deploy-ai-models.sh && chmod 755 /home/modelarts-agent/deploy-ai-models.sh && sh /home/modelarts-agent/deploy-ai-models.sh ${model} ${cards} ${port}' > /home/modelarts-agent/deploy_${model}.log 2>&1 &

3. Dual-machine Deployment

3. 双机部署

User says: "dual-machine deploy model_name head:IP worker:IP port XXXX"

Before dual-machine deploy, both head and worker nodes need
mkdir -p /home/modelarts-agent
.

Head Node Command Template:

bash

nohup bash -c 'export ray_head_ip=${head_ip} && export model_name=${model} && export port=${port} && wget -P /home/modelarts-agent/ https://documentation-samples-17.obs.cn-north-9.myhuaweicloud.com/solution-as-code-publicbucket/solution-as-code-module/quickly-deploy-llm-on-modelarts-lite-devserver/userdata/deploy-large-models/dual-machine/qwen3-235b-a22b.sh && chmod 755 /home/modelarts-agent/qwen3-235b-a22b.sh && sh /home/modelarts-agent/qwen3-235b-a22b.sh head ${head_ip} ${model} ${port}' > /home/modelarts-agent/deploy_${model}_head.log 2>&1 &

Worker Node Command Template:

bash

nohup bash -c 'export ray_head_ip=${head_ip} && export model_name=${model} && export port=${port} && wget -P /home/modelarts-agent/ https://documentation-samples-17.obs.cn-north-9.myhuaweicloud.com/solution-as-code-publicbucket/solution-as-code-module/quickly-deploy-llm-on-modelarts-lite-devserver/userdata/deploy-large-models/dual-machine/qwen3-235b-a22b.sh && chmod 755 /home/modelarts-agent/qwen3-235b-a22b.sh && sh /home/modelarts-agent/qwen3-235b-a22b.sh worker ${head_ip} ${model} ${port}' > /home/modelarts-agent/deploy_${model}_worker.log 2>&1 &

VL Dual-machine Deployment:

For VL models (Qwen3-VL-235B-A22B-Instruct, etc.), use the following scripts:

VL Head Node Command:

bash

nohup bash -c 'export ray_head_ip=${head_ip} && export model_name=${model} && export port=${port} && wget -P /home/modelarts-agent/ https://documentation-samples-17.obs.cn-north-9.myhuaweicloud.com/solution-as-code-publicbucket/solution-as-code-module/quickly-deploy-llm-on-modelarts-lite-devserver/userdata/deploy-vl-model/dual-machine/qwen3-vl-235b-a22b.sh && chmod 755 /home/modelarts-agent/qwen3-vl-235b-a22b.sh && sh /home/modelarts-agent/qwen3-vl-235b-a22b.sh head ${head_ip} ${model} ${port}' > /home/modelarts-agent/deploy_${model}_head.log 2>&1 &

VL Worker Node Command:

bash

nohup bash -c 'export ray_head_ip=${head_ip} && export model_name=${model} && export port=${port} && wget -P /home/modelarts-agent/ https://documentation-samples-17.obs.cn-north-9.myhuaweicloud.com/solution-as-code-publicbucket/solution-as-code-module/quickly-deploy-llm-on-modelarts-lite-devserver/userdata/deploy-vl-model/dual-machine/qwen3-vl-235b-a22b.sh && chmod 755 /home/modelarts-agent/qwen3-vl-235b-a22b.sh && sh /home/modelarts-agent/qwen3-vl-235b-a22b.sh worker ${head_ip} ${model} ${port}' > /home/modelarts-agent/deploy_${model}_worker.log 2>&1 &

用户指令："dual-machine deploy model_name head:IP worker:IP port XXXX"

双机部署前，主节点与工作节点均需执行
mkdir -p /home/modelarts-agent
。

主节点命令模板:

bash

nohup bash -c 'export ray_head_ip=${head_ip} && export model_name=${model} && export port=${port} && wget -P /home/modelarts-agent/ https://documentation-samples-17.obs.cn-north-9.myhuaweicloud.com/solution-as-code-publicbucket/solution-as-code-module/quickly-deploy-llm-on-modelarts-lite-devserver/userdata/deploy-large-models/dual-machine/qwen3-235b-a22b.sh && chmod 755 /home/modelarts-agent/qwen3-235b-a22b.sh && sh /home/modelarts-agent/qwen3-235b-a22b.sh head ${head_ip} ${model} ${port}' > /home/modelarts-agent/deploy_${model}_head.log 2>&1 &

工作节点命令模板:

bash

nohup bash -c 'export ray_head_ip=${head_ip} && export model_name=${model} && export port=${port} && wget -P /home/modelarts-agent/ https://documentation-samples-17.obs.cn-north-9.myhuaweicloud.com/solution-as-code-publicbucket/solution-as-code-module/quickly-deploy-llm-on-modelarts-lite-devserver/userdata/deploy-large-models/dual-machine/qwen3-235b-a22b.sh && chmod 755 /home/modelarts-agent/qwen3-235b-a22b.sh && sh /home/modelarts-agent/qwen3-235b-a22b.sh worker ${head_ip} ${model} ${port}' > /home/modelarts-agent/deploy_${model}_worker.log 2>&1 &

VL双机部署:

对于VL模型（如Qwen3-VL-235B-A22B-Instruct等），使用以下脚本：

VL主节点命令:

bash

nohup bash -c 'export ray_head_ip=${head_ip} && export model_name=${model} && export port=${port} && wget -P /home/modelarts-agent/ https://documentation-samples-17.obs.cn-north-9.myhuaweicloud.com/solution-as-code-publicbucket/solution-as-code-module/quickly-deploy-llm-on-modelarts-lite-devserver/userdata/deploy-vl-model/dual-machine/qwen3-vl-235b-a22b.sh && chmod 755 /home/modelarts-agent/qwen3-vl-235b-a22b.sh && sh /home/modelarts-agent/qwen3-vl-235b-a22b.sh head ${head_ip} ${model} ${port}' > /home/modelarts-agent/deploy_${model}_head.log 2>&1 &

VL工作节点命令:

bash

nohup bash -c 'export ray_head_ip=${head_ip} && export model_name=${model} && export port=${port} && wget -P /home/modelarts-agent/ https://documentation-samples-17.obs.cn-north-9.myhuaweicloud.com/solution-as-code-publicbucket/solution-as-code-module/quickly-deploy-llm-on-modelarts-lite-devserver/userdata/deploy-vl-model/dual-machine/qwen3-vl-235b-a22b.sh && chmod 755 /home/modelarts-agent/qwen3-vl-235b-a22b.sh && sh /home/modelarts-agent/qwen3-vl-235b-a22b.sh worker ${head_ip} ${model} ${port}' > /home/modelarts-agent/deploy_${model}_worker.log 2>&1 &

4. Deployment Confirmation Flow

4. 部署确认流程

Sensitive operation, must show full command and wait for user "confirm" before executing.

After deploy command sent:

Notify user: Ready, starting deployment of ${model}, log at
```
/home/modelarts-agent/deploy_${model}.log
```
Check log every 2 minutes, report progress (loading weights, Dynamo compiling, service starting, etc.)
When port is listening, notify deployment success
Deployment failure handling (strict compliance):
- Deployment failed = Report failure reason, no automatic retry
- Never auto-change image and retry
- Never auto-modify parameters and retry
- Never try other deployment methods
- Only report error, let user decide next step
Output API sample for user:

Deployment successful! ${model} is ready

Service URL: http://${IP}:${PORT}/v1/chat/completions

Example request:
curl -X POST http://${IP}:${PORT}/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"${model}","messages":[{"role":"user","content":"hello"}],"max_tokens":256}'

Multimodal request (if supported):
curl -X POST http://${IP}:${PORT}/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"${model}","messages":[{"role":"user","content":[{"type":"image_url","image_url":{"url":"image_url"}},{"type":"text","text":"describe the image"}]}],"max_tokens":512}'

敏感操作，必须展示完整命令并等待用户确认「confirm」后再执行。

发送部署命令后：

通知用户：准备就绪，开始部署${model}，日志路径为
```
/home/modelarts-agent/deploy_${model}.log
```
每2分钟检查一次日志，汇报进度（加载权重、Dynamo编译、服务启动等）
端口开始监听时，通知部署成功
部署失败处理（严格遵守）:
- 部署失败 = 汇报失败原因，禁止自动重试
- 禁止自动更换镜像重试
- 禁止自动修改参数重试
- 禁止尝试其他部署方式
- 仅汇报错误，由用户决定下一步操作
输出API示例供用户使用：

Deployment successful! ${model} is ready

Service URL: http://${IP}:${PORT}/v1/chat/completions

Example request:
curl -X POST http://${IP}:${PORT}/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"${model}","messages":[{"role":"user","content":"hello"}],"max_tokens":256}'

Multimodal request (if supported):
curl -X POST http://${IP}:${PORT}/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"${model}","messages":[{"role":"user","content":[{"type":"image_url","image_url":{"url":"image_url"}},{"type":"text","text":"describe the image"}]}],"max_tokens":512}'

II. Deployment Log

II. 部署日志

User says: "deployment log model_name"

Agent uses

huawei-cloud-ascend-remote-connect

to execute:

bash

tail -50 /home/modelarts-agent/deploy_${model}.log

用户指令："deployment log model_name"

Agent使用

huawei-cloud-ascend-remote-connect

执行：

bash

tail -50 /home/modelarts-agent/deploy_${model}.log

III. Deployment Status

III. 部署状态

User says: "deployment status port XXXX"

Agent uses

huawei-cloud-ascend-remote-connect

to execute:

bash

ss -tlnp | grep :

Port listening = Service ready for testing.

用户指令："deployment status port XXXX"

Agent使用

huawei-cloud-ascend-remote-connect

执行：

bash

ss -tlnp | grep :

端口处于监听状态 = 服务已就绪可测试。

IV. Test (Execute)

IV. 测试（执行）

User says: "test model_name prompt:xxx" or "test model_name image:URL"

Test flow (strict compliance):

Show full curl command for user to review
Wait for user "confirm" or "send" before executing
Structured result output:

Test Result

| Field | Value |
|-------|-------|
| id | chatcmpl-xxx |
| model | Qwen3-VL-32B-Instruct |
| prompt_tokens | 93 |
| completion_tokens | 400 |
| total_tokens | 493 |
| finish_reason | stop |

Model Response:
[Extract full content, no truncation]

Raw Response:
[Full JSON, no truncation]

用户指令："test model_name prompt:xxx" 或 "test model_name image:URL"

测试流程（严格遵守）:

展示完整curl命令供用户审核
等待用户确认「confirm」或「send」后再执行
结构化输出结果:

测试结果

| 字段 | 值 |
|-------|-------|
| id | chatcmpl-xxx |
| model | Qwen3-VL-32B-Instruct |
| prompt_tokens | 93 |
| completion_tokens | 400 |
| total_tokens | 493 |
| finish_reason | stop |

模型响应:
[提取完整内容，不截断]

原始响应:
[完整JSON，不截断]

LLM Chat Completions

LLM对话补全

bash

curl -s -X POST http://${IP}:${PORT}/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"${model}","messages":[{"role":"user","content":"${prompt}"}],"max_tokens":1024,"temperature":0.7}'

bash

curl -s -X POST http://${IP}:${PORT}/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"${model}","messages":[{"role":"user","content":"${prompt}"}],"max_tokens":1024,"temperature":0.7}'

Multimodal VL

多模态VL

bash

curl -s -X POST http://${IP}:${PORT}/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"${model}","messages":[{"role":"system","content":"You are a helpful assistant."},{"role":"user","content":[{"type":"image_url","image_url":{"url":"${image_url}"}},{"type":"text","text":"${prompt}"}]}],"max_tokens":512,"temperature":0.7}'

bash

curl -s -X POST http://${IP}:${PORT}/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"${model}","messages":[{"role":"system","content":"You are a helpful assistant."},{"role":"user","content":[{"type":"image_url","image_url":{"url":"${image_url}"}},{"type":"text","text":"${prompt}"}]}],"max_tokens":512,"temperature":0.7}'

Embedding

bash

curl -s -X POST http://${IP}:${PORT}/v1/embeddings \
  -H 'Content-Type: application/json' \
  -d '{"model":"${model}","input":"${text}"}'

bash

curl -s -X POST http://${IP}:${PORT}/v1/embeddings \
  -H 'Content-Type: application/json' \
  -d '{"model":"${model}","input":"${text}"}'

Rerank

bash

curl -s -X POST http://${IP}:${PORT}/v1/rerank \
  -H 'Content-Type: application/json' \
  -d '{"model":"${model}","query":"${query}","documents":["${doc1}","${doc2}"]}'

bash

curl -s -X POST http://${IP}:${PORT}/v1/rerank \
  -H 'Content-Type: application/json' \
  -d '{"model":"${model}","query":"${query}","documents":["${doc1}","${doc2}"]}'

V. Write Test Command (Generate Only)

V. 生成测试命令（仅生成）

User says: "write test command model_name prompt:xxx"

Same logic as "test", but only output command text, no execution.

用户指令："write test command model_name prompt:xxx"

逻辑与「测试」相同，但仅输出命令文本，不执行。

API Parameter Manual

API参数手册

LLM Parameters (/v1/chat/completions)

LLM参数（/v1/chat/completions）

Parameter	Required	Default	Description
model	Yes	—	Model name, same as deployment
messages	Yes	—	Message list, each with role and content
max_tokens	No	16	Max generation tokens
temperature	No	1.0	Sampling randomness, 0=greedy
top_p	No	1.0	Nucleus sampling threshold
top_k	No	-1	Only consider top-K tokens
stream	No	false	Streaming output (SSE)
chat_template_kwargs	No	{}	Template params, e.g. {"enable_thinking": false}

参数	是否必填	默认值	描述
model	是	—	模型名称，与部署时一致
messages	是	—	消息列表，每条消息包含角色与内容
max_tokens	否	16	最大生成token数
temperature	否	1.0	采样随机性，0=贪婪采样
top_p	否	1.0	核采样阈值
top_k	否	-1	仅考虑前K个token
stream	否	false	流式输出（SSE）
chat_template_kwargs	否	{}	模板参数，例如{"enable_thinking": false}

VL Extra Parameters

VL额外参数

Parameter	Description
content[]	Array format: image_url object + text object
detail	Image precision: auto/high/low

参数	描述
content[]	数组格式：image_url对象 + text对象
detail	图像精度：auto/high/low

Embedding Parameters (/v1/embeddings)

Embedding参数（/v1/embeddings）

Parameter	Required	Description
model	Yes	Model name
input	Yes	String or string list
encoding_format	No	float/base64

参数	是否必填	描述
model	是	模型名称
input	是	字符串或字符串列表
encoding_format	否	float/base64

Rerank Parameters (/v1/rerank)

Rerank参数（/v1/rerank）

Parameter	Required	Description
model	Yes	Model name
query	Yes	Query text
documents	Yes	Document list to rerank
top_n	No	Return top N

参数	是否必填	描述
model	是	模型名称
query	是	查询文本
documents	是	待重排序的文档列表
top_n	否	返回前N个结果

Execution Mode

执行模式

This skill operates in stateless mode (Rule 2). All context (host, credentials, model info) must be explicitly passed by Agent (Rule 1).

本技能采用无状态模式（规则2）。所有上下文信息（主机、凭证、模型信息）必须由Agent明确传递（规则1）。

Prerequisites (Agent orchestrated)

前提条件（由Agent编排）

Before calling this skill, Agent MUST:

Establish SSH connection using
```
huawei-cloud-ascend-remote-connect
```
- Agent receives: host, port, user, password from user
- Agent validates connection is successful
Check NPU status using
```
huawei-cloud-ascend-command
```
- Agent checks: NPU health, HBM availability
- Agent validates: sufficient cards for model deployment

调用本技能前，Agent必须：

使用
huawei-cloud-ascend-remote-connect
建立SSH连接
- Agent从用户处获取：主机、端口、用户名、密码
- Agent验证连接成功
使用
huawei-cloud-ascend-command
检查NPU状态
- Agent检查：NPU健康状态、HBM可用性
- Agent验证：有足够的卡数用于模型部署

Skill Execution

技能执行

This skill receives explicit parameters from Agent:

bash

undefined

本技能从Agent接收明确参数：

bash

undefined

Model matching (local operation)

模型匹配（本地操作）

python3 scripts/deploy_helper.py match <model_name>

Script URL generation (local operation)

脚本URL生成（本地操作）

python3 scripts/deploy_helper.py script <model_name>

Deploy command generation (local operation)

部署命令生成（本地操作）

python3 scripts/deploy_helper.py command <model> <cards> <port>

undefined

python3 scripts/deploy_helper.py command <model> <cards> <port>

undefined

Remote Deployment Execution

远程部署执行

Agent executes deployment commands on remote server:

bash

undefined

Agent通过SSH在远程服务器执行部署命令：

bash

undefined

Agent uses SSH to execute deployment on DevServer

Agent使用SSH在DevServer上执行部署

ssh root@<host> "cd /path/to/model && bash deploy.sh"

undefined

ssh root@<host> "cd /path/to/model && bash deploy.sh"

undefined

Post-Deployment (Agent orchestrated)

部署后操作（由Agent编排）

After deployment, Agent calls

huawei-cloud-ascend-command

to:

Monitor NPU HBM usage
Check deployment process status
Verify model endpoint is responding

部署完成后，Agent调用

huawei-cloud-ascend-command

：

监控NPU HBM使用率
检查部署进程状态
验证模型端点可正常响应

Parameter Flow

参数流转

User Input                    Agent                      This Skill
    │                          │                            │
    │ host, password           │                            │
    ├─────────────────────────▶│                            │
    │                          │ SSH connect                │
    │                          ├───────────────────────────▶│
    │                          │                            │ (remote-connect)
    │                          │◀───────────────────────────┤
    │                          │                            │
    │                          │ NPU check                  │
    │                          ├───────────────────────────▶│
    │                          │                            │ (ascend-command)
    │                          │◀───────────────────────────┤
    │                          │                            │
    │ model_name, cards        │                            │
    ├─────────────────────────▶│                            │
    │                          │ match model                │
    │                          ├───────────────────────────▶│
    │                          │                            │ deploy_helper.py
    │                          │◀───────────────────────────┤
    │                          │                            │
    │                          │ execute deploy             │
    │                          ├───────────────────────────▶│
    │                          │                            │ (via SSH)
    │                          │◀───────────────────────────┤
    │                          │                            │
    │                          │ monitor NPU                │
    │                          ├───────────────────────────▶│
    │                          │                            │ (ascend-command)
    │                          │◀───────────────────────────┤
    │                          │                            │
    ▼                          ▼                            ▼

Note: No direct skill-to-skill calls. All orchestration by Agent (Rule 3).

用户输入                    Agent                      本技能
    │                          │                            │
    │ host, password           │                            │
    ├─────────────────────────▶│                            │
    │                          │ SSH connect                │
    │                          ├───────────────────────────▶│
    │                          │                            │ (remote-connect)
    │                          │◀───────────────────────────┤
    │                          │                            │
    │                          │ NPU check                  │
    │                          ├───────────────────────────▶│
    │                          │                            │ (ascend-command)
    │                          │◀───────────────────────────┤
    │                          │                            │
    │ model_name, cards        │                            │
    ├─────────────────────────▶│                            │
    │                          │ match model                │
    │                          ├───────────────────────────▶│
    │                          │                            │ deploy_helper.py
    │                          │◀───────────────────────────┤
    │                          │                            │
    │                          │ execute deploy             │
    │                          ├───────────────────────────▶│
    │                          │                            │ (via SSH)
    │                          │◀───────────────────────────┤
    │                          │                            │
    │                          │ monitor NPU                │
    │                          ├───────────────────────────▶│
    │                          │                            │ (ascend-command)
    │                          │◀───────────────────────────┤
    │                          │                            │
    ▼                          ▼                            ▼

注意：技能之间不可直接调用，所有编排由Agent完成（规则3）。

References

参考文档

Document	Description
task-deploy-model.md	Deployment task steps
task-test-model.md	Testing task steps
model-catalog.md	Complete model catalog
api-parameters.md	API parameter reference
prerequisites.md	Prerequisites checklist
verification-method.md	Verification steps
troubleshooting.md	Troubleshooting guide
scripts/deploy_helper.py	Model matching helper

文档	描述
task-deploy-model.md	部署任务步骤
task-test-model.md	测试任务步骤
model-catalog.md	完整模型目录
api-parameters.md	API参数参考
prerequisites.md	前提条件清单
verification-method.md	验证步骤
troubleshooting.md	故障排查指南
scripts/deploy_helper.py	模型匹配工具

huawei-cloud-ascend-models-deploy

Original

Translation

Huawei Cloud Ascend Models Deploy

华为云Ascend模型部署

Overview

概述

Architecture

架构

System Architecture Diagram

系统架构图

Agent Orchestration Flow

Agent编排流程

Related Skills Table

相关技能表

Prerequisites

前提条件

Mandatory Rules (AI Must Follow)

强制规则（AI必须遵守）

Natural Language Understanding Rules

自然语言理解规则

Operation Type Detection

操作类型检测

Information Extraction Rules

信息提取规则

Efficient base64 conversion

Efficient base64 conversion

Supported Machine Types

支持的机型

Model Catalog

模型目录

Large Language Models (LLM) — Endpoint: /v1/chat/completions

大语言模型（LLM） — 端点：/v1/chat/completions

Vision-Language (VL) — Endpoint: /v1/chat/completions

视觉语言模型（VL） — 端点：/v1/chat/completions

Embedding — Endpoint: /v1/embeddings (V0 backend only, single card only)

Embedding模型 — 端点：/v1/embeddings（仅V0后端，仅支持单卡）

Rerank — Endpoint: /v1/rerank (single card only)

Rerank模型 — 端点：/v1/rerank（仅支持单卡）

OpenSource (Multimodal)

开源模型（多模态）

Deploy Script Auto-Match (Must use, never guess script URL)

部署脚本自动匹配（必须使用，禁止猜测脚本URL）

Match model (returns category, script URL, min cards, etc.)

匹配模型（返回类别、脚本URL、最小卡数等）

Generate deploy command directly

直接生成部署命令

List all models (optional category filter)

列出所有模型（可选类别过滤）

Core Commands

核心命令

Operation Flow

操作流程

I. Deployment

I. 部署

1. Pre-deployment Check (Must execute every time, cannot skip)

1. 部署前检查（每次必须执行，不可跳过）

2. Single-machine Deployment

2. 单机部署

3. Dual-machine Deployment

3. 双机部署

4. Deployment Confirmation Flow

4. 部署确认流程

II. Deployment Log

II. 部署日志

III. Deployment Status

III. 部署状态

IV. Test (Execute)

IV. 测试（执行）

LLM Chat Completions

LLM对话补全

Multimodal VL

多模态VL

Embedding

Embedding

Rerank

Rerank

V. Write Test Command (Generate Only)

V. 生成测试命令（仅生成）

API Parameter Manual