huawei-cloud-ascend-small-model-migrate

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Huawei Cloud Ascend Small Model Migration

华为云Ascend小模型迁移

Overview

概述

This skill guides the migration workflow for small vision models to Ascend NPU, covering structure analysis → migration verification → performance optimization.

Architecture: Model Analysis → Environment Setup → NPU Inference → Performance Profiling → Bottleneck Analysis → Optimization Recommendations

Related Skills:

```
huawei-cloud-msmodelslim-model-analysis
```
- Model structure analysis for migration path determination

huawei-cloud-msot-msopprof-operator-profiler

- Operator performance data collection

```
huawei-cloud-ascend-profiler-db-explorer
```
- Profiling database analysis for bottleneck identification
```
huawei-cloud-ascendc-operator-performance-optim
```
- Optional: AscendC operator optimization for bottleneck operators

本技能指导视觉类小模型迁移至Ascend NPU的完整工作流程，涵盖结构分析→迁移验证→性能优化环节。

流程架构：模型分析→环境搭建→NPU推理→性能剖析→瓶颈分析→优化建议

关联技能:

```
huawei-cloud-msmodelslim-model-analysis
```
- 模型结构分析，用于确定迁移路径

huawei-cloud-msot-msopprof-operator-profiler

- 算子性能数据采集

```
huawei-cloud-ascend-profiler-db-explorer
```
- 剖析数据库分析，用于识别瓶颈
```
huawei-cloud-ascendc-operator-performance-optim
```
- 可选：针对瓶颈算子的AscendC算子优化

Architecture Components

架构组件

This skill involves the following cloud services and components:

Ascend NPU: Target hardware for model deployment (Ascend 910B series)
torch_npu: PyTorch adapter for Ascend NPU
MSProf: Ascend profiling tool for performance analysis
Ultralytics: YOLO model framework support
Docker: Container environment for consistent deployment

本技能涉及以下云服务与组件：

Ascend NPU: 模型部署的目标硬件（Ascend 910B系列）
torch_npu: 适配Ascend NPU的PyTorch适配器
MSProf: 用于性能分析的Ascend剖析工具
Ultralytics: YOLO模型框架支持
Docker: 用于一致性部署的容器环境

Use Cases

使用场景

Typical Problem Scenarios:

Migrating vision models from GPU to Ascend NPU
Deploying YOLO/ResNet/UNet models on Ascend hardware
Optimizing small model performance on NPU
Verifying model accuracy after migration
Identifying performance bottlenecks in computer vision models

Typical User Phrases:

"Migrate YOLOv8 to Ascend NPU
"How to run ResNet on Ascend?
"Optimize UNet inference on NPU
"Verify model accuracy after migration
"Analyze performance bottlenecks in my vision model
"YOLOModelMigrationAscendNPU
"AscendModel？
"ModelMigrationNPU？

典型问题场景:

将视觉模型从GPU迁移至Ascend NPU
在Ascend硬件上部署YOLO/ResNet/UNet模型
优化小模型在NPU上的性能
验证迁移后的模型精度
识别计算机视觉模型中的性能瓶颈

典型用户表述:

"将YOLOv8迁移至Ascend NPU
"如何在Ascend上运行ResNet？
"优化UNet在NPU上的推理性能
"验证迁移后的模型精度
"分析我的视觉模型的性能瓶颈
"YOLOModelMigrationAscendNPU
"AscendModel？
"ModelMigrationNPU？

Scope

适用范围

Supported:

Encoder-only architectures (ResNet, VGG, EfficientNet)
Detection models (YOLO, Faster-RCNN, SSD)
Segmentation models (UNet, DeepLab)
Other non-Decoder-only LLM models

Not supported:

Decoder-only LLM (Qwen, LLaMA, DeepSeek) - requires adapter-based quantization approach
Understanding VLM text backbone - requires adapter-based quantization approach

支持的模型类型:

仅编码器架构（ResNet、VGG、EfficientNet）
检测模型（YOLO、Faster-RCNN、SSD）
分割模型（UNet、DeepLab）
其他非仅解码器LLM模型

不支持的模型类型:

仅解码器LLM（Qwen、LLaMA、DeepSeek）- 需要基于适配器的量化方案
多模态大模型文本骨干网络 - 需要基于适配器的量化方案

Workflow

工作流程

┌─────────────────────────────────────────────────────────────┐
│  Step 1: Model Structure Analysis                           │
│  → Determine msmodelslim compatibility                      │
│  → Output structure analysis + migration path suggestion    │
└─────────────────────────────────────────────────────────────┘
                              ↓
┌─────────────────────────────────────────────────────────────┐
│  Step 2: Environment Preparation + Migration Verification  │
│  → Configure torch_npu environment                         │
│  → Run inference test                                      │
│  → Verify accuracy                                         │
└─────────────────────────────────────────────────────────────┘
                              ↓
┌─────────────────────────────────────────────────────────────┐
│  Step 3: Performance Data Collection                       │
│  → Collect operator performance data                       │
│  → Output performance data location                        │
└─────────────────────────────────────────────────────────────┘
                              ↓
┌─────────────────────────────────────────────────────────────┐
│  Step 4: Performance Analysis                              │
│  → Analyze profiling data for bottlenecks                  │
│  → Output complete operator time distribution              │
│  → Identify bottleneck and well-performing operators       │
└─────────────────────────────────────────────────────────────┘
                              ↓
┌─────────────────────────────────────────────────────────────┐
│  Step 5: Optimization Suggestions                          │
│  → Provide optimization solutions for bottleneck operators │
│  → Optional operator optimization                          │
└─────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────┐
│  Step 1: Model Structure Analysis                           │
│  → Determine msmodelslim compatibility                      │
│  → Output structure analysis + migration path suggestion    │
└─────────────────────────────────────────────────────────────┘
                              ↓
┌─────────────────────────────────────────────────────────────┐
│  Step 2: Environment Preparation + Migration Verification  │
│  → Configure torch_npu environment                         │
│  → Run inference test                                      │
│  → Verify accuracy                                         │
└─────────────────────────────────────────────────────────────┘
                              ↓
┌─────────────────────────────────────────────────────────────┐
│  Step 3: Performance Data Collection                       │
│  → Collect operator performance data                       │
│  → Output performance data location                        │
└─────────────────────────────────────────────────────────────┘
                              ↓
┌─────────────────────────────────────────────────────────────┐
│  Step 4: Performance Analysis                              │
│  → Analyze profiling data for bottlenecks                  │
│  → Output complete operator time distribution              │
│  → Identify bottleneck and well-performing operators       │
└─────────────────────────────────────────────────────────────┘
                              ↓
┌─────────────────────────────────────────────────────────────┐
│  Step 5: Optimization Suggestions                          │
│  → Provide optimization solutions for bottleneck operators │
│  → Optional operator optimization                          │
└─────────────────────────────────────────────────────────────┘

Step 1: Model Structure Analysis

步骤1：模型结构分析

1.1 Analysis Process

1.1 分析流程

Read the model configuration to analyze:

Model implementation source (transformers or local directory)
Architecture type (Decoder-only / Encoder-only / Encoder-Decoder)
Layer-by-layer loading requirements
MoE fused weight risks

读取模型配置以分析以下内容：

模型实现来源（transformers库或本地目录）
架构类型（仅解码器/仅编码器/编码器-解码器）
逐层加载需求
MoE融合权重风险

1.2 Migration Path Determination

1.2 迁移路径确定

Architecture Type	Recommended Path
Decoder-only LLM	Adapter-based quantization
Understanding VLM text backbone	Adapter-based quantization
Encoder-only / Detection / Segmentation	Continue with this skill
Other	Manual determination required

架构类型	推荐路径
仅解码器LLM	基于适配器的量化方案
多模态大模型文本骨干网络	基于适配器的量化方案
仅编码器/检测/分割模型	继续使用本技能
其他	需要手动确定

1.3 Output Analysis Report

1.3 输出分析报告

markdown

undefined

markdown

undefined

Model Structure Analysis Result

模型结构分析结果

Basic Information

基本信息

Model Name: xxx
Architecture Type: Encoder-only / Decoder-only / Encoder-Decoder
Parameter Count: xxx
Source: transformers / local directory

模型名称：xxx
架构类型：仅编码器/仅解码器/编码器-解码器
参数数量：xxx
来源：transformers库/本地目录

Support Status

支持状态

msmodelslim Support: Yes/No
Recommended Migration Path: torch_npu direct migration / msmodelslim adaptation

msmodelslim支持：是/否
推荐迁移路径：torch_npu直接迁移/msmodelslim适配

Migration Suggestions

迁移建议

[Specific recommendations]

---

[具体建议内容]

---

Step 2: Environment Preparation + Migration Verification

步骤2：环境准备 + 迁移验证

2.1 Default Verification Environment

2.1 默认验证环境

Server: ascend-server-01
Container: skill-the
Image: quay.io/ascend/vllm-ascend:v0.18.0
NPU: 8× Ascend 910B3

服务器: ascend-server-01
容器: skill-the
镜像: quay.io/ascend/vllm-ascend:v0.18.0
NPU: 8× Ascend 910B3

2.2 Environment Configuration

2.2 环境配置

bash

undefined

bash

undefined

Enter container

docker exec -it skill-the bash

Install dependencies

pip install torch_npu pip install ultralytics # For YOLO series

Or other model-specific libraries

OpenCV dependencies (if needed)

apt install libgl1 libglib2.0-0

undefined

apt install libgl1 libglib2.0-0

undefined

2.3 Migration Verification Script

2.3 迁移验证脚本

python

import torch
import torch_npu

python

import torch
import torch_npu

Check NPU availability

print(f"NPU available: {torch.npu.is_available()}") print(f"NPU count: {torch.npu.device_count()}")

Load model

model = ... # Model loading code model = model.to('npu:0')

Inference test

with torch.no_grad(): output = model(input_tensor)

print(f"Inference success: {output is not None}")

undefined

with torch.no_grad(): output = model(input_tensor)

print(f"Inference success: {output is not None}")

undefined

2.4 Output Migration Verification Report

2.4 输出迁移验证报告

markdown

undefined

markdown

undefined

Migration Verification Result

迁移验证结果

Environment Information

环境信息

Server: ascend-server-01
Container: skill-the
torch_npu Version: xxx
NPU Status: Normal

服务器: ascend-server-01
容器: skill-the
torch_npu版本: xxx
NPU状态: 正常

Inference Test

推理测试

Model Loading: Success/Failure
NPU Inference: Success/Failure
Accuracy Verification: Pass/Fail

模型加载: 成功/失败
NPU推理: 成功/失败
精度验证: 通过/未通过

Performance Metrics

性能指标

Average Inference Time: xxx ms
FPS: xxx

---

平均推理时间: xxx ms
FPS: xxx

---

Step 3: Performance Data Collection

步骤3：性能数据采集

3.1 Performance Collection Process

3.1 性能采集流程

Collect operator performance data using the profiling skill:

On-board collection (device mode)
Or simulation collection (simulator mode)

使用剖析技能采集算子性能数据：

板载采集（设备模式）
或模拟采集（模拟器模式）

3.2 Output Performance Data Location

3.2 输出性能数据位置

markdown

undefined

markdown

undefined

Performance Collection Result

性能采集结果

Data Location

数据位置

Server: ascend-server-01
Path: /home/xxx/PROF_xxx/
Database: msprof_xxx.db
Collection Time: xxx

服务器: ascend-server-01
路径: /home/xxx/PROF_xxx/
数据库: msprof_xxx.db
采集时间: xxx

Collection Configuration

采集配置

Mode: device / simulator
NPU: npu:0
Collection Duration: xxx s

---

模式: device / simulator
NPU: npu:0
采集时长: xxx s

---

Step 4: Performance Analysis

步骤4：性能分析

4.1 Analysis Process

4.1 分析流程

Query and analyze:

Top N operator time consumption
Group statistics by operator type
AI_CPU / AI_CORE / AI_VECTOR_CORE distribution

查询并分析以下内容：

耗时Top N的算子
按算子类型分组统计
AI_CPU / AI_CORE / AI_VECTOR_CORE分布情况

4.2 SQL Query Example

4.2 SQL查询示例

sql

-- Top 20 operators by time
SELECT op_name, op_type, total_time, call_times
FROM op_summary
ORDER BY total_time DESC
LIMIT 20;

-- Group by type
SELECT op_type, SUM(total_time) as type_time, COUNT(*) as count
FROM op_summary
GROUP BY op_type
ORDER BY type_time DESC;

sql

-- Top 20 operators by time
SELECT op_name, op_type, total_time, call_times
FROM op_summary
ORDER BY total_time DESC
LIMIT 20;

-- Group by type
SELECT op_type, SUM(total_time) as type_time, COUNT(*) as count
FROM op_summary
GROUP BY op_type
ORDER BY type_time DESC;

4.3 Output Performance Analysis Report

4.3 输出性能分析报告

markdown

undefined

markdown

undefined

Performance Analysis Result

性能分析结果

Operator Time Distribution (TOP 20)

算子耗时分布（TOP 20）

Rank	Operator Name	Type	Time(ms)	Percentage	Call Count
1	xxx	AI_CPU	xxx	xx%	xxx
...	...	...	...	...	...

排名	算子名称	类型	耗时(ms)	占比	调用次数
1	xxx	AI_CPU	xxx	xx%	xxx
...	...	...	...	...	...

Statistics by Type

按类型统计

Type	Total Time	Percentage	Operator Count
AI_CPU	xxx	xx%	xxx
AI_CORE	xxx	xx%	xxx
AI_VECTOR_CORE	xxx	xx%	xxx

类型	总耗时	占比	算子数量
AI_CPU	xxx	xx%	xxx
AI_CORE	xxx	xx%	xxx
AI_VECTOR_CORE	xxx	xx%	xxx

Bottleneck Operators (>5% usage)

瓶颈算子（占比>5%）

Operator	Type	Percentage	Issue
xxx	AI_CPU	xx%	[Specific issue]

算子	类型	占比	问题描述
xxx	AI_CPU	xx%	[具体问题]

Well-performing Operators

性能优良算子

Operator	Type	Description
Conv2D	AI_CORE	High Cube utilization, normal

---

算子	类型	描述
Conv2D	AI_CORE	Cube利用率高，运行正常

---

Step 5: Optimization Suggestions

步骤5：优化建议

5.1 Common Bottlenecks and Solutions

5.1 常见瓶颈与解决方案

Bottleneck Type	Cause	Optimization Solution
Index operator high time	AI_CPU implementation	Develop optimized version with AscendC
TransData high time	Format conversion overhead	Reduce CPU-NPU data transfer
NMS fallback to CPU	Operator not NPU supported	Develop NPU version NMS with AscendC
Upsample slow	Vector core efficiency	Optimize upsample operator

瓶颈类型	原因	优化方案
Index算子耗时高	AI_CPU实现	使用AscendC开发优化版本
TransData耗时高	格式转换开销	减少CPU-NPU数据传输
NMS回退至CPU	算子未支持NPU	使用AscendC开发NPU版本NMS
Upsample速度慢	向量核心效率低	优化Upsample算子

5.2 Output Optimization Suggestions Report

5.2 输出优化建议报告

markdown

undefined

markdown

undefined

Optimization Suggestions

优化建议

Priority Ranking

优先级排序

Priority	Operator	Issue	Solution	Expected Gain
P0	xxx	xxx	xxx	xx%
P1	xxx	xxx	xxx	xx%

优先级	算子	问题	解决方案	预期收益
P0	xxx	xxx	xxx	xx%
P1	xxx	xxx	xxx	xx%

Next Steps

后续步骤

[Specific optimization steps]
Operator optimization may be performed for bottleneck operators

---

[具体优化步骤]
可针对瓶颈算子进行算子优化

---

Complete Report Template

完整报告模板

After completing each migration task, output a complete report:

markdown

undefined

完成每项迁移任务后，输出完整报告：

markdown

undefined

[Model Name] Ascend Migration Report

[模型名称] Ascend迁移报告

1. Model Structure Analysis

1. 模型结构分析

[Step 1 output]

[步骤1输出内容]

2. Migration Verification

2. 迁移验证

[Step 2 output]

[步骤2输出内容]

3. Performance Collection

3. 性能采集

[Step 3 output]

[步骤3输出内容]

4. Performance Analysis

4. 性能分析

[Step 4 output]

[步骤4输出内容]

5. Optimization Suggestions

5. 优化建议

[Step 5 output]

[步骤5输出内容]

Summary

总结

Migration Status: Success/Failure
Inference Performance: xxx ms / xxx FPS
Main Bottlenecks: xxx
Optimization Direction: xxx

---

迁移状态：成功/失败
推理性能：xxx ms / xxx FPS
主要瓶颈：xxx
优化方向：xxx

---

Default Environment

默认环境

Server: ascend-server-01:22 (root/Hhuawei@smb)
Container: skill-the
NPU: 8× Ascend 910B3 (64G HBM each)
CANN: cann-version-placeholder.220

服务器: ascend-server-01:22 (root/Hhuawei@smb)
容器: skill-the
NPU: 8× Ascend 910B3（单卡64G HBM）
CANN: cann-version-placeholder.220

Prerequisites

前置条件

System Requirements

系统要求

Python 3.8+
torch_npu >= 2.0.0
msprof >= 7.0.0
ultralytics >= 8.0.0 (for YOLO models)

Python 3.8+
torch_npu >= 2.0.0
msprof >= 7.0.0
ultralytics >= 8.0.0（针对YOLO模型）

Environment Check

环境检查

Prerequisite check: Python3 + torch_npu + msprof required

bash

python3 --version  # Python3 >= 3.8
python3 -c "import torch_npu; print('OK')"  # NPU PyTorch support
python3 -c "import msprof; print('OK')"  # Profiling library

If not installed:

pip3 install --user torch_npu msprof ultralytics

前置检查：需安装Python3 + torch_npu + msprof

bash

python3 --version  # Python3 >= 3.8
python3 -c "import torch_npu; print('OK')"  # NPU PyTorch支持
python3 -c "import msprof; print('OK')"  # 剖析库

若未安装：

pip3 install --user torch_npu msprof ultralytics

Additional System Dependencies

额外系统依赖

For computer vision models:

bash

apt install libgl1 libglib2.0-0  # OpenCV dependencies

针对计算机视觉模型：

bash

apt install libgl1 libglib2.0-0  # OpenCV依赖

Enhanced Features

增强功能

Performance Baseline Comparison Module

性能基线对比模块

This skill includes a performance baseline comparison mechanism that compares current model performance against industry-standard baselines:

Features:

Pre-defined Baselines: Baseline data for common models (YOLOv8, ResNet50, UNet, EfficientNet) on Ascend NPU
Delta Analysis: Generates performance gap analysis and optimization potential assessment
Performance Ranking: Compares against similar models in the benchmark database
Trend Analysis: Tracks performance improvements across migration iterations

Baseline Database:

Model	Batch Size	Latency (ms)	Throughput (FPS)	Accuracy
YOLOv8n	32	2.3	434	53.1% mAP
YOLOv8s	16	4.8	208	60.6% mAP
ResNet50	64	1.2	533	76.1% top-1
UNet	8	8.5	94	-

Delta Analysis Output:

markdown

undefined

本技能包含性能基线对比机制，可将当前模型性能与行业标准基线进行对比：

功能特性:

预定义基线: 常见模型（YOLOv8、ResNet50、UNet、EfficientNet）在Ascend NPU上的基线数据
差值分析: 生成性能差距分析与优化潜力评估
性能排名: 与基准数据库中的同类模型进行对比
趋势分析: 跟踪多轮迁移迭代中的性能提升情况

基线数据库:

模型	批量大小	延迟(ms)	吞吐量(FPS)	精度
YOLOv8n	32	2.3	434	53.1% mAP
YOLOv8s	16	4.8	208	60.6% mAP
ResNet50	64	1.2	533	76.1% top-1
UNet	8	8.5	94	-

差值分析输出:

markdown

undefined

Performance Baseline Comparison

性能基线对比

Target Model: YOLOv8s
Baseline Reference: YOLOv8s @ Ascend 910B

目标模型: YOLOv8s
基线参考: YOLOv8s @ Ascend 910B

Performance Gap

性能差距

Metric	Current	Baseline	Gap
Latency	5.2 ms	4.8 ms	+8.3%
Throughput	192 FPS	208 FPS	-7.7%
Accuracy	60.2%	60.6%	-0.4%

指标	当前值	基线值	差距
延迟	5.2 ms	4.8 ms	+8.3%
吞吐量	192 FPS	208 FPS	-7.7%
精度	60.2%	60.6%	-0.4%

Optimization Potential

优化潜力

Priority P0: Reduce latency by optimizing Conv operators
Priority P1: Improve memory access pattern
Expected Gain: ~10-15% performance improvement

undefined

优先级P0: 通过优化Conv算子降低延迟
优先级P1: 改进内存访问模式
预期收益: ~10-15%性能提升

undefined

Resource Estimation & Planning Tool

资源估算与规划工具

This skill provides pre-migration resource estimation capabilities:

Features:

Memory Requirements Prediction: Estimates NPU memory usage based on model size and batch configuration
Inference Time Estimation: Predicts latency and throughput before deployment
Batch Size Recommendation: Suggests optimal batch size based on target latency constraints
Multi-card Scaling Guidance: Provides scaling recommendations for multi-device deployment
Cost-Benefit Analysis: Evaluates optimization investment vs. expected performance gain

Resource Estimation Output:

markdown

undefined

本技能提供迁移前资源估算能力：

功能特性:

内存需求预测: 根据模型大小与批量配置估算NPU内存占用
推理时间估算: 在部署前预测延迟与吞吐量
批量大小建议: 根据目标延迟约束建议最优批量大小
多卡扩展指导: 提供多设备部署的扩展建议
成本收益分析: 评估优化投入与预期性能收益

资源估算输出:

markdown

undefined

Resource Estimation Report

资源估算报告

Model: YOLOv8s
Input Resolution: 640x640

模型: YOLOv8s
输入分辨率: 640x640

Memory Requirements

内存需求

Component	Size
Model Weights	21 MB
Activation (BS=16)	480 MB
Total Estimated	501 MB

组件	大小
模型权重	21 MB
激活值（BS=16）	480 MB
总估算值	501 MB

Performance Prediction

性能预测

Batch Size	Estimated Latency	Estimated Throughput
8	3.2 ms	250 FPS
16	4.8 ms	208 FPS
32	8.5 ms	188 FPS

批量大小	估算延迟	估算吞吐量
8	3.2 ms	250 FPS
16	4.8 ms	208 FPS
32	8.5 ms	188 FPS

Recommended Configuration

Reference Documents

参考文档

Document	Description
Acceptance Criteria	Functional and non-functional acceptance criteria
Verification Method	Step-by-step verification guide
Troubleshooting	Common issues and solutions
Report Template	Report generation template
Profiler SQL	SQL query references
Migration Scripts	Migration helper scripts

文档	描述
验收标准	功能与非功能验收标准
验证方法	分步验证指南
故障排查	常见问题与解决方案
报告模板	报告生成模板
剖析器SQL	SQL查询参考
迁移脚本	迁移辅助脚本

Prerequisites (Duplicate - See Above)

前置条件（重复 - 见上文）

torch_npu >= 2.0.0 installed
msprof >= 7.0.0 installed
Ascend NPU environment configured
Model code to be migrated

torch_npu >= 2.0.0已安装
msprof >= 7.0.0已安装
Ascend NPU环境已配置
待迁移的模型代码

Core Commands

核心命令

bash

undefined

bash

undefined

Analyze model migration feasibility

分析模型迁移可行性

python3 scripts/analyze_model.py --model /path/to/model

Verify NPU inference

验证NPU推理

python3 scripts/verify_npu.py --model /path/to/model --input test.jpg

undefined

python3 scripts/verify_npu.py --model /path/to/model --input test.jpg

undefined

Parameter Confirmation

参数确认

Parameter	Description	Required
model	Model code path	Yes
input	Test input data	Yes
output	Output directory	No

参数	描述	是否必填
model	模型代码路径	是
input	测试输入数据	是
output	输出目录	否