Optimizing Model Structures for HarmonyOS Next

Developing efficient and performant AI models for resource-constrained devices like those running HarmonyOS Next requires careful optimization of model architecture. This article delves into practical techniques for reducing computational complexity and the number of parameters in your models, thereby enhancing performance and resource utilization.

I. Principles and Objectives of Model Structure Optimization

(1) Basic Principles

Model structure optimization in HarmonyOS Next is analogous to designing an efficient bridge: we aim for maximum stability (performance) with minimal materials (parameters) and construction effort (computational complexity). The core principle involves scrutinizing the model's network structure to identify redundant components or areas ripe for improvement. For example, in a Convolutional Neural Network (CNN), excessive filters in convolutional layers or an overabundance of neurons in fully connected layers represent potential optimization targets.

(2) Main Objectives

1. Reducing Computational Complexity: Lower computational complexity directly translates to faster model execution on HarmonyOS Next devices. Optimizing the model structure to reduce unnecessary computations is akin to streamlining a car's engine for increased speed. Reducing multiplications in convolutional layers or simplifying the logic of fully connected layers significantly improves inference speed – crucial for real-time applications like object detection in security systems.
2. Reducing the Number of Parameters: Fewer parameters mean a smaller model footprint, conserving valuable storage space on devices with limited resources. This is comparable to decluttering a warehouse, freeing up space and improving overall efficiency. A smaller parameter count also reduces memory requirements during training and inference, contributing to greater system stability.

(3) The Mechanism of Improving Model Performance

Reducing computational complexity directly reduces the time and resources needed for data processing. For instance, in an image classification model, reducing convolutional layer computations accelerates image classification, improving response time. Reducing the number of parameters conserves storage, but also mitigates overfitting. Overfitting occurs when a model learns training data details too well, ignoring overall data characteristics. Fewer parameters encourage focus on key features, improving the model's generalization ability and performance on unseen data.

II. Common Structure Optimization Technologies and Applications

(1) Pruning Technology

1. Principles and Application Methods: Pruning is like trimming a tree, removing branches (neurons or connections) that minimally contribute to the overall structure and function. In HarmonyOS Next, neuron or connection importance is determined – often by calculating neuron activity. Low-activity neurons, having minimal impact on model output, are candidates for pruning. A common approach involves calculating the output change of each neuron across various input data. Neurons showing minimal output change indicate low activity. These are pruned based on a defined threshold. Careful selection of the pruning ratio is crucial; too high a ratio may drastically reduce performance, while too low a ratio yields minimal gains.
2. Application Advantages Combined with the Characteristics of HarmonyOS Next: HarmonyOS Next prioritizes efficient resource use and device adaptability. Pruned models, with reduced parameters and memory footprint, are better suited for devices with storage and memory limitations. For instance, a pruned gesture recognition model on a smartwatch loads and runs swiftly, improving user experience.

(2) Quantization Technology

1. Principles and Application Methods: Quantization is like replacing a high-precision tool (e.g., a millimeter ruler) with a less precise but more efficient one (e.g., a centimeter ruler). In models, this means converting high-precision parameters (e.g., 32-bit floating-point numbers) to lower-precision data types (e.g., 8-bit integers). This requires careful consideration of data distribution and precision loss. Parameter distribution is analyzed to determine quantization range and step size. For example, weight parameters in an image classification model ranging from -1 to 1 can be quantized to 8-bit integers, converting floating-point numbers to integers via a mapping function. Accuracy impact, especially in high-accuracy scenarios (medical image diagnosis), must be carefully considered when selecting a quantization strategy.
2. Application Advantages Combined with the Characteristics of HarmonyOS Next: HarmonyOS Next supports diverse hardware platforms. Quantized models exhibit better compatibility due to low-precision data types' computational and storage efficiency, making the best use of hardware capabilities. On low-end devices, processors optimized for low-precision calculations allow faster inference with quantized models.

(3) Comparison of Optimization Effects of Different Technologies in Different Model Structures (in Table Form)

III. Case Analysis and Performance Evaluation

(1) Case Selection and Background

We use the VGG-16 image classification CNN as a case study. While accurate, its large parameter count and high computational complexity make it unsuitable for direct deployment on HarmonyOS Next devices. Our goal is to optimize VGG-16 for efficient execution on HarmonyOS Next devices, such as enabling fast image classification on smartphones, while maintaining acceptable accuracy.

(2) Application Process of Structural Optimization Technologies

1. Pruning Operation: We begin by pruning VGG-16. Analyzing neuron activity reveals that some neurons in the final fully connected layers exhibit low activity. After setting a suitable pruning threshold, these low-activity neurons are removed. For instance, pruning approximately 50% of neurons in the last fully connected layer significantly reduces parameters, but accuracy drops from 92% to 88%.
2. Quantization Processing: Next, the pruned model undergoes quantization. Appropriate quantization range and step size are selected based on parameter distribution, converting 32-bit floating-point parameters to 8-bit integers. This further reduces model size and improves computational efficiency, but accuracy decreases to 85%.
3. Fine-tuning Training: To recover accuracy, we fine-tune the quantized model. Using the original training dataset but reducing training epochs to avoid overfitting, we achieve an accuracy of 90%, a slight decrease from the original but within acceptable limits. Parameter count is reduced by approximately 70%, and computational complexity is substantially decreased.

(3) Performance Evaluation Indicators and Results

1. Accuracy Evaluation: Pre-optimization accuracy was 92%; post-optimization, it reached 90%. This is acceptable for many image classification tasks. Identifying common objects (animals, plants, everyday items) at 90% accuracy ensures high recognition correctness.
2. Inference Speed Evaluation: Testing on HarmonyOS Next smartphones shows that pre-optimization, average inference time per image was 1.5 seconds. Post-optimization, this dropped to approximately 0.5 seconds, a 2x improvement due to reduced computational complexity and hardware-level optimization of low-precision calculations.
3. Memory Occupation Evaluation: Pre-optimization memory usage was around 300MB; post-optimization, it reduced to approximately 90MB, benefiting devices with limited resources by enabling simultaneous execution of more applications or processing of larger images.

(4) Experience Summary and Precautions

1. Experience Summary: Combining pruning and quantization effectively reduces model parameters and computational complexity, with fine-tuning restoring accuracy to an acceptable level. Pruning threshold and quantization parameters require careful selection and iterative adjustment based on model characteristics and application needs. For real-time applications, higher pruning ratios can be used, followed by more extensive fine-tuning to balance accuracy and speed.
2. Precautions:
   • Accuracy Monitoring: Continuously monitor accuracy during optimization. Adjust the strategy or increase fine-tuning if accuracy drops excessively.
   • Hardware Adaptation: Account for device-specific hardware capabilities (computing power, storage structure) when applying optimization techniques. Low-end devices may require tailored optimization or specific quantization strategies.
   • Impact of Data Distribution: Quantization is sensitive to data distribution. Analyze parameter distributions thoroughly before quantization to ensure appropriate quantization range and step size, minimizing accuracy loss.