Huawei Cloud Ascend Small Model Migrate

Huawei Cloud Ascend Small Model Migration

Overview

This skill guides the migration workflow for small vision models to Ascend NPU, covering structure analysis → migration verification → performance optimization.

Architecture: Model Analysis → Environment Setup → NPU Inference → Performance Profiling → Bottleneck Analysis → Optimization Recommendations

Related Skills:

• huawei-cloud-msmodelslim-model-analysis - Model structure analysis for migration path determination
• huawei-cloud-msot-msopprof-operator-profiler - Operator performance data collection
• huawei-cloud-ascend-profiler-db-explorer - Profiling database analysis for bottleneck identification
• huawei-cloud-ascendc-operator-performance-optim - Optional: AscendC operator optimization for bottleneck operators

Architecture Components

This skill involves the following cloud services and components:

• Ascend NPU: Target hardware for model deployment (Ascend 910B series)
• torch_npu: PyTorch adapter for Ascend NPU
• MSProf: Ascend profiling tool for performance analysis
• Ultralytics: YOLO model framework support
• Docker: Container environment for consistent deployment

Use Cases

Typical Problem Scenarios:

• Migrating vision models from GPU to Ascend NPU
• Deploying YOLO/ResNet/UNet models on Ascend hardware
• Optimizing small model performance on NPU
• Verifying model accuracy after migration
• Identifying performance bottlenecks in computer vision models

Typical User Phrases:

• "Migrate YOLOv8 to Ascend NPU
• "How to run ResNet on Ascend?
• "Optimize UNet inference on NPU
• "Verify model accuracy after migration
• "Analyze performance bottlenecks in my vision model
• "YOLOModelMigrationAscendNPU
• "AscendModel？
• "ModelMigrationNPU？

Scope

Supported:

• Encoder-only architectures (ResNet, VGG, EfficientNet)
• Detection models (YOLO, Faster-RCNN, SSD)
• Segmentation models (UNet, DeepLab)
• Other non-Decoder-only LLM models

Not supported:

• Decoder-only LLM (Qwen, LLaMA, DeepSeek) - requires adapter-based quantization approach
• Understanding VLM text backbone - requires adapter-based quantization approach

Workflow

┌─────────────────────────────────────────────────────────────┐
│  Step 1: Model Structure Analysis                           │
│  → Determine msmodelslim compatibility                      │
│  → Output structure analysis + migration path suggestion    │
└─────────────────────────────────────────────────────────────┘
                              ↓
┌─────────────────────────────────────────────────────────────┐
│  Step 2: Environment Preparation + Migration Verification  │
│  → Configure torch_npu environment                         │
│  → Run inference test                                      │
│  → Verify accuracy                                         │
└─────────────────────────────────────────────────────────────┘
                              ↓
┌─────────────────────────────────────────────────────────────┐
│  Step 3: Performance Data Collection                       │
│  → Collect operator performance data                       │
│  → Output performance data location                        │
└─────────────────────────────────────────────────────────────┘
                              ↓
┌─────────────────────────────────────────────────────────────┐
│  Step 4: Performance Analysis                              │
│  → Analyze profiling data for bottlenecks                  │
│  → Output complete operator time distribution              │
│  → Identify bottleneck and well-performing operators       │
└─────────────────────────────────────────────────────────────┘
                              ↓
┌─────────────────────────────────────────────────────────────┐
│  Step 5: Optimization Suggestions                          │
│  → Provide optimization solutions for bottleneck operators │
│  → Optional operator optimization                          │
└─────────────────────────────────────────────────────────────┘

---

Step 1: Model Structure Analysis

1.1 Analysis Process

Read the model configuration to analyze:

• Model implementation source (transformers or local directory)
• Architecture type (Decoder-only / Encoder-only / Encoder-Decoder)
• Layer-by-layer loading requirements
• MoE fused weight risks

1.2 Migration Path Determination

1.3 Output Analysis Report

## Model Structure Analysis Result

### Basic Information
- Model Name: xxx
- Architecture Type: Encoder-only / Decoder-only / Encoder-Decoder
- Parameter Count: xxx
- Source: transformers / local directory

### Support Status
- msmodelslim Support: Yes/No
- Recommended Migration Path: torch_npu direct migration / msmodelslim adaptation

### Migration Suggestions
[Specific recommendations]

---

Step 2: Environment Preparation + Migration Verification

2.1 Default Verification Environment

• Server: ascend-server-01
• Container: skill-the
• Image: quay.io/ascend/vllm-ascend:v0.18.0
• NPU: 8× Ascend 910B3

2.2 Environment Configuration

# Enter container
docker exec -it skill-the bash

# Install dependencies
pip install torch_npu
pip install ultralytics  # For YOLO series
# Or other model-specific libraries

# OpenCV dependencies (if needed)
apt install libgl1 libglib2.0-0

2.3 Migration Verification Script

import torch
import torch_npu

# Check NPU availability
print(f"NPU available: {torch.npu.is_available()}")
print(f"NPU count: {torch.npu.device_count()}")

# Load model
model = ...  # Model loading code
model = model.to('npu:0')

# Inference test
with torch.no_grad():
    output = model(input_tensor)

print(f"Inference success: {output is not None}")

2.4 Output Migration Verification Report

## Migration Verification Result

### Environment Information
- Server: ascend-server-01
- Container: skill-the
- torch_npu Version: xxx
- NPU Status: Normal

### Inference Test
- Model Loading: Success/Failure
- NPU Inference: Success/Failure
- Accuracy Verification: Pass/Fail

### Performance Metrics
- Average Inference Time: xxx ms
- FPS: xxx

---

Step 3: Performance Data Collection

3.1 Performance Collection Process

Collect operator performance data using the profiling skill:

• On-board collection (device mode)
• Or simulation collection (simulator mode)

3.2 Output Performance Data Location

## Performance Collection Result

### Data Location
- Server: ascend-server-01
- Path: /home/xxx/PROF_xxx/
- Database: msprof_xxx.db
- Collection Time: xxx

### Collection Configuration
- Mode: device / simulator
- NPU: npu:0
- Collection Duration: xxx s

---

Step 4: Performance Analysis

4.1 Analysis Process

Query and analyze: 1. Top N operator time consumption 2. Group statistics by operator type 3. AI_CPU / AI_CORE / AI_VECTOR_CORE distribution

4.2 SQL Query Example

-- Top 20 operators by time
SELECT op_name, op_type, total_time, call_times
FROM op_summary
ORDER BY total_time DESC
LIMIT 20;

-- Group by type
SELECT op_type, SUM(total_time) as type_time, COUNT(*) as count
FROM op_summary
GROUP BY op_type
ORDER BY type_time DESC;

4.3 Output Performance Analysis Report

## Performance Analysis Result

### Operator Time Distribution (TOP 20)
| Rank | Operator Name | Type | Time(ms) | Percentage | Call Count |
|------|--------------|------|----------|------------|------------|
| 1 | xxx | AI_CPU | xxx | xx% | xxx |
| ... | ... | ... | ... | ... | ... |

### Statistics by Type
| Type | Total Time | Percentage | Operator Count |
|------|------------|------------|----------------|
| AI_CPU | xxx | xx% | xxx |
| AI_CORE | xxx | xx% | xxx |
| AI_VECTOR_CORE | xxx | xx% | xxx |

### Bottleneck Operators (>5% usage)
| Operator | Type | Percentage | Issue |
|----------|------|------------|-------|
| xxx | AI_CPU | xx% | [Specific issue] |

### Well-performing Operators
| Operator | Type | Description |
|----------|------|-------------|
| Conv2D | AI_CORE | High Cube utilization, normal |

---

Step 5: Optimization Suggestions

5.1 Common Bottlenecks and Solutions

5.2 Output Optimization Suggestions Report

## Optimization Suggestions

### Priority Ranking
| Priority | Operator | Issue | Solution | Expected Gain |
|----------|----------|-------|----------|---------------|
| P0 | xxx | xxx | xxx | xx% |
| P1 | xxx | xxx | xxx | xx% |

### Next Steps
1. [Specific optimization steps]
2. Operator optimization may be performed for bottleneck operators

---

Complete Report Template

After completing each migration task, output a complete report:

# [Model Name] Ascend Migration Report

## 1. Model Structure Analysis
[Step 1 output]

## 2. Migration Verification
[Step 2 output]

## 3. Performance Collection
[Step 3 output]

## 4. Performance Analysis
[Step 4 output]

## 5. Optimization Suggestions
[Step 5 output]

## Summary
- Migration Status: Success/Failure
- Inference Performance: xxx ms / xxx FPS
- Main Bottlenecks: xxx
- Optimization Direction: xxx

---

Default Environment

• Server: ascend-server-01:22 (root/Hhuawei@smb)
• Container: skill-the
• NPU: 8× Ascend 910B3 (64G HBM each)
• CANN: cann-version-placeholder.220

Prerequisites

System Requirements

• Python 3.8+
• torch_npu >= 2.0.0
• msprof >= 7.0.0
• ultralytics >= 8.0.0 (for YOLO models)

Environment Check

Prerequisite check: Python3 + torch_npu + msprof required

```bash

python3 --version # Python3 >= 3.8

python3 -c "import torch_npu; print('OK')" # NPU PyTorch support

python3 -c "import msprof; print('OK')" # Profiling library

```

If not installed: pip3 install --user torch_npu msprof ultralytics

Additional System Dependencies

For computer vision models:

apt install libgl1 libglib2.0-0  # OpenCV dependencies

Enhanced Features

Performance Baseline Comparison Module

This skill includes a performance baseline comparison mechanism that compares current model performance against industry-standard baselines:

Features:

• Pre-defined Baselines: Baseline data for common models (YOLOv8, ResNet50, UNet, EfficientNet) on Ascend NPU
• Delta Analysis: Generates performance gap analysis and optimization potential assessment
• Performance Ranking: Compares against similar models in the benchmark database
• Trend Analysis: Tracks performance improvements across migration iterations

Baseline Database:

Delta Analysis Output:

## Performance Baseline Comparison
- Target Model: YOLOv8s
- Baseline Reference: YOLOv8s @ Ascend 910B

### Performance Gap
| Metric | Current | Baseline | Gap |
|--------|---------|----------|-----|
| Latency | 5.2 ms | 4.8 ms | +8.3% |
| Throughput | 192 FPS | 208 FPS | -7.7% |
| Accuracy | 60.2% | 60.6% | -0.4% |

### Optimization Potential
- Priority P0: Reduce latency by optimizing Conv operators
- Priority P1: Improve memory access pattern
- Expected Gain: ~10-15% performance improvement

Resource Estimation & Planning Tool

This skill provides pre-migration resource estimation capabilities:

Features:

• Memory Requirements Prediction: Estimates NPU memory usage based on model size and batch configuration
• Inference Time Estimation: Predicts latency and throughput before deployment
• Batch Size Recommendation: Suggests optimal batch size based on target latency constraints
• Multi-card Scaling Guidance: Provides scaling recommendations for multi-device deployment
• Cost-Benefit Analysis: Evaluates optimization investment vs. expected performance gain

Resource Estimation Output:

## Resource Estimation Report
- Model: YOLOv8s
- Input Resolution: 640x640

### Memory Requirements
| Component | Size |
|-----------|------|
| Model Weights | 21 MB |
| Activation (BS=16) | 480 MB |
| Total Estimated | 501 MB |

### Performance Prediction
| Batch Size | Estimated Latency | Estimated Throughput |
|------------|-------------------|---------------------|
| 8 | 3.2 ms | 250 FPS |
| 16 | 4.8 ms | 208 FPS |
| 32 | 8.5 ms | 188 FPS |

### Recommended Configuration
- Optimal Batch Size: 16
- Target Latency: 4.8 ms
- Expected Throughput: 208 FPS
- Memory Utilization: ~78% of 64GB HBM

Reference Documents

Prerequisites (Duplicate - See Above)

• torch_npu >= 2.0.0 installed
• msprof >= 7.0.0 installed
• Ascend NPU environment configured
• Model code to be migrated

Core Commands

# Analyze model migration feasibility
python3 scripts/analyze_model.py --model /path/to/model

# Verify NPU inference
python3 scripts/verify_npu.py --model /path/to/model --input test.jpg

Parameter Confirmation

Acceptance Criteria

Functional Acceptance Criteria

1. Model Structure Analysis

2. Environment Setup

3. Migration Verification

4. Performance Profiling

Correct/Error Pattern Comparison

Server Connection

Correct: Verify server accessibility first

ssh -p 22 root@ascend-server-01 "echo connected"
ssh -p 22 root@ascend-server-01 "npu-smi info"

Error: Start migration without server verification

# Skipping connection test leads to timeout errors later
docker exec skill-the bash -c "pip install ..."

NPU Detection

Correct: Verify NPU availability

import torch
print(f"NPU available: {torch.npu.is_available()}")
print(f"NPU count: {torch.npu.device_count()}")

Error: Assume NPU is available

# Without checking availability
model = model.to('npu:0')  # May fail silently

Model Loading

Correct: Load model with proper device mapping

import torch
import torch_npu

model = YourModel()
model = model.to('npu:0')
model.eval()

Error: Load without NPU conversion

model = YourModel()  # Stays on CPU
output = model(input)  # No NPU acceleration

Non-Functional Acceptance Criteria

Test Cases Summary

Positive Test Cases

1. TC-001: Encoder-only model (ResNet) migration 2. TC-002: Detection model (YOLO) migration 3. TC-003: Segmentation model (UNet) migration 4. TC-004: End-to-end workflow execution 5. TC-005: Performance profiling and analysis

Negative Test Cases

1. TC-N01: Unsupported model type (Decoder-only LLM) 2. TC-N02: NPU not available on server 3. TC-N03: Container not running 4. TC-N04: Missing dependencies 5. TC-N05: Profiling data collection timeout

MigrationVerificationScriptsReference

YOLO systemcolumnVerification

#!/usr/bin/env python3
"""YOLO AscendMigrationVerificationScripts"""

import torch
import torch_npu
from ultralytics import YOLO
import time

def check_npu():
    """Check NPU Environment"""
    print("=" * 50)
    print("NPU EnvironmentCheck")
    print("=" * 50)
    print(f"torch_npu Version: {torch_npu.__version__}")
    print(f"NPU canuse: {torch.npu.is_available()}")
    print(f"NPU numberamount: {torch.npu.device_count()}")
    for i in range(torch.npu.device_count()):
        print(f"  NPU {i}: {torch.npu.get_device_name(i)}")
    print()

def test_inference(model_path, image_path, device='npu:0'):
    """pushmanageTesting"""
    print("=" * 50)
    print("pushmanageTesting")
    print("=" * 50)
    
    # LoadModel
    print(f"LoadModel: {model_path}")
    model = YOLO(model_path)
    
    # pushmanage
    print(f"pushmanagefigureslice: {image_path}")
    print(f"Device: {device}")
    
    # prepassionate
    model(image_path, device=device)
    
    # PerformanceTesting
    times = []
    for i in range(100):
        start = time.time()
        result = model(image_path, device=device)
        times.append((time.time() - start) * 1000)
    
    avg_time = sum(times) / len(times)
    min_time = min(times)
    max_time = max(times)
    fps = 1000 / avg_time
    
    print()
    print("PerformanceResult:")
    print(f"  averageaverageTime consumption: {avg_time:.2f} ms")
    print(f"  mostsmallTime consumption: {min_time:.2f} ms")
    print(f"  mostlargeTime consumption: {max_time:.2f} ms")
    print(f"  FPS: {fps:.1f}")
    print()
    
    # checktestResult
    print("checktestResult:")
    for r in result:
        for box in r.boxes:
            cls = int(box.cls[0])
            conf = float(box.conf[0])
            print(f"  typecategory {cls}: placeinformationdegree {conf:.2f}")
    print()
    
    return {
        'avg_time': avg_time,
        'fps': fps,
        'success': True
    }

if __name__ == '__main__':
    check_npu()
    result = test_inference('yolov8n.pt', 'bus.jpg')
    print(f"MigrationVerification: {'becomefunction' if result['success'] else 'lossfailure'}")

ResNet systemcolumnVerification

#!/usr/bin/env python3
"""ResNet AscendMigrationVerificationScripts"""

import torch
import torch_npu
import torchvision.models as models
from torchvision import transforms
from PIL import Image
import time

def check_npu():
    """Check NPU Environment"""
    print("=" * 50)
    print("NPU EnvironmentCheck")
    print("=" * 50)
    print(f"NPU canuse: {torch.npu.is_available()}")
    print(f"NPU numberamount: {torch.npu.device_count()}")
    print()

def test_inference(model_name='resnet50', image_path='test.jpg'):
    """pushmanageTesting"""
    device = 'npu:0'
    
    # LoadModel
    print(f"LoadModel: {model_name}")
    model = getattr(models, model_name)(pretrained=True)
    model = model.to(device)
    model.eval()
    
    # prehandlemanage
    preprocess = transforms.Compose([
        transforms.Resize(256),
        transforms.CenterCrop(224),
        transforms.ToTensor(),
        transforms.Normalize(mean=[0.485, 0.456, 0.406], 
                           std=[0.229, 0.224, 0.225]),
    ])
    
    # Loadfigureslice
    image = Image.open(image_path).convert('RGB')
    input_tensor = preprocess(image).unsqueeze(0).to(device)
    
    # prepassionate
    with torch.no_grad():
        _ = model(input_tensor)
    
    # PerformanceTesting
    torch.npu.synchronize()
    times = []
    for _ in range(100):
        start = time.time()
        with torch.no_grad():
            output = model(input_tensor)
        torch.npu.synchronize()
        times.append((time.time() - start) * 1000)
    
    avg_time = sum(times) / len(times)
    fps = 1000 / avg_time
    
    print(f"averageaverageTime consumption: {avg_time:.2f} ms")
    print(f"FPS: {fps:.1f}")
    
    # pretestResult
    _, pred = torch.max(output, 1)
    print(f"pretesttypecategory: {pred.item()}")

if __name__ == '__main__':
    check_npu()
    test_inference()

throughuseVerificationTemplate

#!/usr/bin/env python3
"""throughuseModelVerificationTemplate"""

import torch
import torch_npu
import time

def verify_model(model_class, model_args, input_shape, device='npu:0'):
    """
    throughuseVerificationfunctionnumber
    
    Args:
        model_class: Modeltype
        model_args: ModelInitializationparameternumber
        input_shape: outputinputshapestatus (batch, channels, height, width)
        device: Device
    """
    print("=" * 50)
    print("ModelVerification")
    print("=" * 50)
    
    # Check NPU
    print(f"NPU canuse: {torch.npu.is_available()}")
    print(f"Device: {device}")
    
    # LoadModel
    model = model_class(**model_args)
    model = model.to(device)
    model.eval()
    print(f"ModelLoadbecomefunction")
    
    # Createoutputinput
    input_tensor = torch.randn(input_shape).to(device)
    
    # prepassionate
    with torch.no_grad():
        _ = model(input_tensor)
    
    # PerformanceTesting
    torch.npu.synchronize()
    times = []
    for _ in range(100):
        start = time.time()
        with torch.no_grad():
            output = model(input_tensor)
        torch.npu.synchronize()
        times.append((time.time() - start) * 1000)
    
    avg_time = sum(times) / len(times)
    min_time = min(times)
    max_time = max(times)
    fps = 1000 / avg_time
    
    print()
    print("PerformanceResult:")
    print(f"  averageaverageTime consumption: {avg_time:.2f} ms")
    print(f"  mostsmallTime consumption: {min_time:.2f} ms")
    print(f"  mostlargeTime consumption: {max_time:.2f} ms")
    print(f"  FPS: {fps:.1f}")
    
    return {
        'avg_time': avg_time,
        'fps': fps,
        'success': True
    }

Performance Analysis SQL QueryReference

CommonQuery

1. OperatorTime consumption TOP N

-- TOP 20 Time consumptionOperator
SELECT 
    op_name,
    op_type,
    total_time / 1000 as total_time_ms,
    total_time * 100.0 / (SELECT SUM(total_time) FROM op_summary) as percent,
    call_times
FROM op_summary
ORDER BY total_time DESC
LIMIT 20;

2. according toOperatortypetypestatisticscalculate

-- Group by Typestatisticscalculate
SELECT 
    op_type,
    SUM(total_time) / 1000 as total_time_ms,
    SUM(total_time) * 100.0 / (SELECT SUM(total_time) FROM op_summary) as percent,
    COUNT(*) as op_count
FROM op_summary
GROUP BY op_type
ORDER BY SUM(total_time) DESC;

3. AI_CPU Operator (potentialinBottleneck)

-- AI_CPU Operatorcolumntable
SELECT 
    op_name,
    total_time / 1000 as total_time_ms,
    call_times
FROM op_summary
WHERE op_type = 'AI_CPU'
ORDER BY total_time DESC;

4. adjustusetimenumbermostmultipleofOperator

-- highfrequencyadjustuseOperator
SELECT 
    op_name,
    op_type,
    call_times,
    total_time / call_times / 1000 as avg_time_us
FROM op_summary
ORDER BY call_times DESC
LIMIT 20;

5. SingleTime consumptionmostlengthofOperator

-- SingleTime consumption TOP
SELECT 
    op_name,
    op_type,
    total_time / call_times / 1000 as avg_time_us,
    call_times
FROM op_summary
WHERE call_times > 0
ORDER BY total_time / call_times DESC
LIMIT 20;

6. Conv2D Operatordetailedsituation

-- Conv2D Performance Analysis
SELECT 
    op_name,
    total_time / 1000 as total_time_ms,
    call_times,
    total_time / call_times / 1000 as avg_time_us
FROM op_summary
WHERE op_name LIKE '%Conv%' OR op_name LIKE '%conv%'
ORDER BY total_time DESC;

7. MatMul/GEMM Operator

-- rulematrixmultiplyOperator
SELECT 
    op_name,
    op_type,
    total_time / 1000 as total_time_ms,
    call_times
FROM op_summary
WHERE op_name LIKE '%MatMul%' OR op_name LIKE '%GEMM%' OR op_name LIKE '%matmul%'
ORDER BY total_time DESC;

8. DatatransferoutputOperator

-- TransData/formatformulaconvertexchangeOperator
SELECT 
    op_name,
    op_type,
    total_time / 1000 as total_time_ms,
    call_times
FROM op_summary
WHERE op_name LIKE '%TransData%' OR op_name LIKE '%trans%' OR op_name LIKE '%Cast%'
ORDER BY total_time DESC;

---

AnalysisReportGenerateQuery

completeadjustPerformancegeneralview

-- Performancegeneralview
SELECT 
    'totalOperatornumber' as metric,
    COUNT(*) as value
FROM op_summary
UNION ALL
SELECT 
    'Total time(ms)' as metric,
    SUM(total_time) / 1000 as value
FROM op_summary
UNION ALL
SELECT 
    'AI_CPUOperatornumber' as metric,
    COUNT(*) as value
FROM op_summary WHERE op_type = 'AI_CPU'
UNION ALL
SELECT 
    'AI_COREOperatornumber' as metric,
    COUNT(*) as value
FROM op_summary WHERE op_type = 'AI_CORE'
UNION ALL
SELECT 
    'AI_CPUTime consumptionoccupycompare(%)' as metric,
    SUM(total_time) * 100.0 / (SELECT SUM(total_time) FROM op_summary) as value
FROM op_summary WHERE op_type = 'AI_CPU';

BottleneckOperatorrecognizecategory

-- occupycompare > 5% ofOperator
SELECT 
    op_name,
    op_type,
    total_time / 1000 as total_time_ms,
    ROUND(total_time * 100.0 / (SELECT SUM(total_time) FROM op_summary), 2) as percent,
    CASE 
        WHEN op_type = 'AI_CPU' THEN 'needOptimization: AI_CPUImplementation'
        WHEN op_name LIKE '%TransData%' THEN 'needOptimization: formatformulaconvertexchangeopenconsume'
        WHEN op_name LIKE '%nms%' OR op_name LIKE '%NMS%' THEN 'needOptimization: NMSFallbackCPU'
        ELSE 'waitAnalysis'
    END as suggestion
FROM op_summary
WHERE total_time * 100.0 / (SELECT SUM(total_time) FROM op_summary) > 5
ORDER BY total_time DESC;

---

tableStructureReference

op_summary table

otherotherCommontable

• task_summary - Tasklevelstatisticscalculate
• model_summary - Modellevelstatisticscalculate
• step_summary - step levelstatisticscalculate

MigrationReportTemplate

completeadjustReportStructure

# [ModelName] AscendMigrationReport

> Generation Time: YYYY-MM-DD HH:MM
> Servicesadapter: ascend-server-01
> Container: skill-the

---

## one, ModelStructureAnalysis

### 1.1 Basicinformationinformation

| itemitem | value |
|------|-----|
| ModelName | xxx |
| ModelSource | transformers / thisregionDirectory / PyPI |
| Architecturetypetype | Encoder-only / Decoder-only / Encoder-Decoder |
| parameternumberamount | xxx M |
| outputinputscaleinch | xxx × xxx |

### 1.2 ArchitectureAnalysis

- **maininterferenetworknetwork: ** ResNet / CSPDarknet / ViT / ...
- **checktesthead: ** have/no
- **Notesforcemachinemake: ** have/no
- **specialspecialOperator: ** NMS / ROIAlign / ...

### 1.3 msmodelslim Compatibility

| Checkitem | Result |
|--------|------|
| Decoder-only Architecture | is/whether |
| Transformers Implementation | is/whether |
| msmodelslim Support | ✅ / ❌ |

### 1.4 MigrationpathlineSuggest

**pushrecommendpathline: ** torch_npu straightconnectMigration

**manageby: **
- [toolbodyreasoncause]

**substituterepresentmethodcase: **
- [ifhave]

---

## two, MigrationVerification

### 2.1 Environmentinformationinformation

| itemitem | value |
|------|-----|
| Servicesadapter | ascend-server-01 |
| Container | skill-the |
| mirrorlike | quay.io/ascend/vllm-ascend:v0.18.0 |
| CANN | cann-version-placeholder.220 |
| torch_npu | x.x.x |
| NPU | 8× Ascend 910B3 |

### 2.2 accordingdependInstallation

pip install torch_npu pip install [Modelaccordingdepend]

### 2.3 pushmanageTestingResult

| itemitem | Result |
|------|------|
| ModelLoad | ✅ becomefunction |
| NPU pushmanage | ✅ becomefunction |
| precisiondegreeVerification | ✅ throughexceed |

### 2.4 Performancefingerstandard

| fingerstandard | value |
|------|-----|
| averageaveragepushmanageTime consumption | xx.xx ms |
| mostsmallTime consumption | xx.xx ms |
| mostlargeTime consumption | xx.xx ms |
| FPS | xx.x |

### 2.5 issueappearofaskproblem

| askproblem | shadowloud | statusstate |
|------|------|------|
| xxx | xxx | waitOptimization |

---

## three, PerformanceCollection

### 3.1 CollectionConfiguration

| itemitem | value |
|------|-----|
| Collectionmodelformula | device / simulator |
| NPU | npu:0 |
| Collectiontimelength | xx s |
| Collectiontimebetween | YYYY-MM-DD HH:MM |

### 3.2 Databitplace

| itemitem | value |
|------|-----|
| Servicesadapter | ascend-server-01 |
| DataDirectory | /home/xxx/PROF_xxx/ |
| Database | msprof_xxx.db |
| Datalargesmall | xx MB |

---

## four, Performance Analysis

### 4.1 OperatorTime consumptiondistributearrange (TOP 20) 

| arrangename | Operatorname | typetype | Time consumption(ms) | occupycompare | adjustusetimenumber | averageaverageTime consumption(us) |
|------|--------|------|----------|------|----------|--------------|
| 1 | xxx | AI_CPU | xx.xx | xx.x% | xxx | xx.xx |
| 2 | xxx | AI_CORE | xx.xx | xx.x% | xxx | xx.xx |
| ... | ... | ... | ... | ... | ... | ... |

### 4.2 according toOperatortypetypestatisticscalculate

| typetype | Total time(ms) | occupycompare | Operatornumber |
|------|------------|------|--------|
| AI_CPU | xx.xx | xx.x% | xxx |
| AI_CORE | xx.xx | xx.x% | xxx |
| AI_VECTOR_CORE | xx.xx | xx.x% | xxx |

### 4.3 BottleneckOperatorAnalysis

| Operator | typetype | occupycompare | askproblemAnalysis |
|------|------|------|----------|
| Index | AI_CPU | xx.x% | searchleadoperateworkin CPU Implementation, validratelow |
| TransData | AI_VECTOR_CORE | xx.x% | formatformulaconvertexchangeopenconsume, needdecreasefew CPU-NPU Datacomereturn |
| NMS | AI_CPU | xx.x% | torchvision::nms notSupport NPU, Fallback CPU |

### 4.4 tableappeargoodgoodOperator

| Operator | typetype | Description |
|------|------|------|
| Conv2D | AI_CORE | Cube utilizeuserate 70-90%, positiveoften |
| BatchNorm | AI_CORE | mergematchto Conv, noamountexternalopenconsume |

### 4.5 PerformanceBottleneckSummary

**mainneedBottleneck: **
1. xxx Operator (occupycompare xx%) : [reasoncause]
2. xxx Operator (occupycompare xx%) : [reasoncause]

**timeneedBottleneck: **
1. xxx Operator (occupycompare xx%) : [reasoncause]

---

## five, OptimizationSuggest

### 5.1 Optimizationmethodcase

| optimizefirstlevel | Operator | askproblem | Optimizationmethodcase | preperiodreceiveadvantageous | difficultdegree |
|--------|------|------|----------|----------|------|
| P0 | Index | AI_CPU Implementation | use AscendC DevelopmentOptimizationversion | +xx% | middle |
| P1 | TransData | formatformulaconvertexchange | decreasefew CPU-NPU Datacomereturn | +xx% | low |
| P2 | NMS | Fallback CPU | use AscendC Development NPU version | +xx% | high |

### 5.2 underonesteptravelmove

1. **briefperiodOptimization**
   - [toolbodySteps]

2. **middleperiodOptimization**
   - canadjustuse `ascendc-operator-performance-optim` DevelopmentOptimizationOperator
   - [toolbodySteps]

3. **lengthperiodOptimization**
   - [toolbodySteps]

---

## Summary

### Migrationstatusstate

✅ **Migrationbecomefunction**

### Performancetableappear

| fingerstandard | value |
|------|-----|
| pushmanageTime consumption | xx.xx ms |
| FPS | xx.x |
| mainneedBottleneck | xxx |

### Optimizationpotentialforce

throughexceedOptimizationBottleneckOperator, precalculatecanliftupgrade **xx%** Performance. 

### RelatedResource

- PerformanceData: `/home/xxx/PROF_xxx/`
- VerificationScripts: `references/migration-scripts.md`
- SQL Query: `references/profiler-sql.md`

---

simpletransformReportTemplate

suitable forrapidspeedVerificationScenarios:

# [ModelName] rapidspeedVerificationReport

## Environment
- Servicesadapter: ascend-server-01 (skill-the)
- NPU: Ascend 910B3

## Result
- Migrationstatusstate: ✅ becomefunction
- pushmanageTime consumption: xx.xx ms
- FPS: xx.x

## Bottleneck
- [mainneedBottleneckOperator]

## underonestep
- [OptimizationSuggest]

Troubleshooting

1. Server Connection Issues

Issue: SSH connection timeout

Symptom: Connection timed out

Solution:

# Verify server IP and port
ping -c 3 ascend-server-01

# Check SSH service
ssh -p 22 root@ascend-server-01 "systemctl status sshd"

# Try with key-based auth
ssh -i /path/to/key -p 22 root@ascend-server-01

Issue: Container not running

Symptom: Error: No such container: skill-the

Solution:

# Check all containers
docker ps -a | grep skill

# Start container if exists but stopped
docker start skill-the

# Or create new container
docker run -itd --name skill-the quay.io/ascend/vllm-ascend:v0.18.0

2. NPU Issues

Issue: NPU not detected

Symptom: torch.npu.is_available() returns False

Solution:

# Check NPU driver
npu-smi info

# Verify CANN installation
pip list | grep ascendl

# Reinstall torch_npu
pip uninstall torch_npu
pip install torch_npu

Issue: NPU memory exhausted

Symptom: RuntimeError: NPU out of memory

Solution:

# Reduce batch size
batch_size = 1

# Clear cache
torch.npu.empty_cache()

# Use gradient checkpointing
model.gradient_checkpointing_enable()

3. Model Loading Issues

Issue: Model architecture not supported

Symptom: RuntimeError: Unsupported operator

Solution:

# Check model requirements
cat /path/to/model/config.json | grep model_type

# Verify torch_npu supports the operators
python3 -c "import torch_npu; print(torch_npu.list_supported_ops())"

Issue: Model weights incompatible

Symptom: RuntimeError: Expected tensor for argument #1

Solution:

# Reload weights
model.load_state_dict(torch.load('model.pt'), strict=False)

# Or convert to NPU format
model = model.to('cpu')
model = model.to('npu:0')

4. Dependency Issues

Issue: pip install fails

Symptom: ERROR: Could not find a version that satisfies the requirement

Solution:

# Update pip
pip install --upgrade pip

# Use specific version
pip install torch_npu==2.0.0

# Or install from source
pip install --no-cache-dir torch_npu

Issue: Version conflict

Symptom: ERROR: torch 2.x is installed but torch_npu requires torch x.x

Solution:

# Check installed versions
pip list | grep torch

# Install compatible versions
pip install torch==2.1.0 torch_npu==2.1.0

5. Performance Profiling Issues

Issue: OPPROF directory not created

Symptom: ls: cannot access 'OPPROF_*': No such file or directory

Solution:

# Run profiling command
msprof op --output=./profiling_data python3 inference.py

# Check permissions
chmod 755 /path/to/output_dir

# Run with full path
cd /path/to/model && msprof op --output=./profiling_data python3 inference.py

Issue: Profiling data incomplete

Symptom: Missing CSV files in OPPROF directory

Solution:

# Run with default metrics
msprof op --aic-metrics=Default --output=./profiling_data python3 inference.py

# Check collection duration
# Add warmup if too short

Quick Diagnostic Commands

# Check server status
ssh -p 22 root@ascend-server-01 "uptime"

# Check NPU status
ssh -p 22 root@ascend-server-01 "npu-smi info"

# Check container logs
docker logs skill-the

# Check Python environment
docker exec skill-the bash -c "pip list | grep torch"

# Test NPU inference
docker exec skill-the bash -c "python3 -c 'import torch; print(torch.npu.is_available())'"

Verification Methods

Prerequisite Verification

1. Verify Server Connectivity

# Test SSH connection
ssh -p 22 root@ascend-server-01 "echo connected"

# Verify NPU availability
ssh -p 22 root@ascend-server-01 "npu-smi info"

# Check container status
ssh -p 22 root@ascend-server-01 "docker ps | grep skill-the"

2. Verify Python Environment

# Inside container
docker exec skill-the bash -c "python3 --version"
docker exec skill-the bash -c "pip list | grep torch"

# Check torch_npu
docker exec skill-the bash -c "python3 -c 'import torch; print(torch.npu.is_available())'"

3. Verify Model Files

# Check model directory
ssh -p 22 root@ascend-server-01 "ls -la /path/to/model/"

# Verify model can be loaded
docker exec skill-the bash -c "python3 -c 'import torch; m=torch.load(\"/path/to/model.pt\"); print(type(m))'"

Functional Verification

1. Model Structure Analysis

# Step 1: Call msmodelslim-model-analysis skill
# Read model config
cat /path/to/model/config.json

# Analyze architecture
python3 analyze_model.py --model-path /path/to/model --output analysis_report.md

2. Environment Setup Verification

# Enter container
docker exec -it skill-the bash

# Install dependencies
pip install torch_npu
pip install ultralytics  # For YOLO

# Verify installation
python3 -c "import torch_npu; print('torch_npu OK')"

3. Migration Verification

# Inside container
import torch
import torch_npu

# Check NPU
print(f"NPU available: {torch.npu.is_available()}")

# Load and migrate model
model = ...  # Your model loading code
model = model.to('npu:0')
model.eval()

# Run inference
with torch.no_grad():
    output = model(input_tensor)

print(f"Inference success: {output is not None}")

4. Performance Profiling

# Collect performance data
cd /path/to/model
msprof op --output=./profiling_data python3 inference.py

# Analyze results
ls -la profiling_data/
cat profiling_data/OpBasicInfo.csv

End-to-End Verification Script

#!/bin/bash
set -e

SERVER="ascend-server-01"
PORT=22
CONTAINER="skill-the"

echo "=== 1. Verify Server Connectivity ==="
ssh -p ${PORT} root@${SERVER} "echo connected"

echo "=== 2. Verify NPU ==="
ssh -p ${PORT} root@${SERVER} "npu-smi info"

echo "=== 3. Verify Container ==="
ssh -p ${PORT} root@${SERVER} "docker ps | grep ${CONTAINER}"

echo "=== 4. Verify Python Environment ==="
docker exec ${CONTAINER} bash -c "python3 --version"
docker exec ${CONTAINER} bash -c "python3 -c 'import torch; print(torch.npu.is_available())'"

echo "=== 5. Test Model Migration ==="
docker exec ${CONTAINER} bash -c "python3 -c 'import torch_npu; print(\"NPU OK\")'"

echo "=== All verifications passed ==="

Verification Checklist