Zeyu Ji

Training large Deep Convolutional Neural Networks (DCNNs) with increasingly large datasets to improve model accuracy has become extremely time-consuming. Distributed training methods, such as data parallelism (DP) and pipeline model parallelism (PMP), offer potential solutions but face challenges like load imbalance and significant communication overhead. This paper introduces GroPipe, a novel architecture that synergistically integrates PMP and DP, markedly improving training speeds. GroPipe employs an automatic model partitioning algorithm based on a performance projection technique, ensuring load balance and facilitating quantitative performance evaluation in PMP. Additionally, it adopts a group-based delayed asynchronous communication strategy to efficiently reduce communication overhead in DP. Using the ResNet and VGG models with the ImageNet dataset, extensive experiments are performed on an 8-GPU server and demonstrate GroPipe’s effectiveness. GroPipe achieves substantial improvements in time to accuracy, showing an average improvement of 42.2% and 14.0% on the ResNet series, and 79.2% and 43.9% on the VGG series, without compromising Top-1 accuracy.

Automated quantization has emerged as an entirely new design paradigm to automate the optimal configuration of bitwidth for deep neural networks (DNNs), making the DNN more memory-efficient and faster to execute on hardware with limited resources. Reinforcement learning (RL) and differentiable neural architecture search (DNAS) are two main solution paths that have shown their superiority. Yet, there are countless methods with various implementations within each path. It has been hard to comprehend their differences and make a relatively fair comparison due to the lack of a benchmark framework and a clear analysis of which aspects are common, respectively distinct, between different implementations. To this end, we introduce BenQ to pave the way towards fair comparisons in two separate race tracks, i.e., intra-comparison of the RL-based and the DNAS-based methods, respectively. We provide a systematic approach, which helps to reveal relatively vital aspects of different implementations. Finally, we conduct comprehensive experi-ments on VGG, AlexNet, ResNet, GoogleNet, MobileNet-V2, and Vision Transformer (ViT), and the new observations shed light on potential future directions for automated quantization to move forward.

University English is one of the basic core courses for all university students. The course involves many difficult concepts and grammatical structures, so it is difficult to most students. The starting point in any teaching program is to determine whether teaching is needed to specify what that teaching should accomplish. So the students’ need is of vital importance to the success of university English teaching. Traditional university English teaching does not take into account students’ individual learning abilities and feedback, which would cause students to not grasp enough the key knowledge, and then make students lose their interest in English. The development of artificial intelligence technology is becoming more and more mature, especially in the application of English teaching, which promotes the reform, development, and modernization of English teaching. Hence, in this paper, we propose a hierarchical teaching method for the university English teaching platform and employ artificial intelligence to find the needs of university students and know about the mastering knowledge of students. Initially, the dataset is preprocessed using normalization, and then, the feature extraction is performed using principal component analysis (PCA). For classification of the data, we employ the -means clustering algorithm. To enhance the evaluation system, the whale optimization algorithm (WOA) is used. The performance of the proposed system is analyzed, and it is shown that the average score of students who used the proposed platform to learn university English is far higher than those students who did not use the platform. Hence, the platform can improve students’ autonomous learning and English abilities.

Due to the increase in computing power, it is possible to improve the feature extraction and data fitting capabilities of DNN networks by increasing their depth and model complexity. However, the big data and complex models greatly increase the training overhead of DNN, so accelerating their training process becomes a key task. The Tianhe-3 peak speed is designed to target E-class, and the huge computing power provides a potential opportunity for DNN training. We implement and extend LeNet, AlexNet, VGG, and ResNet model training for a single MT-2000+ and FT-2000+ compute nodes, as well as extended multi-node clusters, and propose an improved gradient synchronization process for Dynamic Allreduce communication optimization strategy for the gradient synchronization process base on the ARM architecture features of the Tianhe-3 prototype, providing experimental data and theoretical basis for further enhancing and improving the performance of the Tianhe-3 prototype in large-scale distributed training of neural networks.

Fluid mechanical simulation is a typical high-performance computing problem. Due to the development of high-precision parallel algorithms, traditional computing platforms are unable to satisfy the computing requirements of large-scale algorithms. The Sunway TaihuLight supercomputer, which uses the SW26010 processor as its computing node, provides a powerful computing performance for this purpose. In this paper, the Sunway hierarchical parallel fluid machinery (swHPFM) framework and algorithm are proposed. Using the proposed framework and algorithm, engineers can exploit the parallelism of the existing fluid mechanical algorithm and achieve a satisfactory performance on the Sunway TaihuLight. In the framework, a suitable mapping of the model and the system architecture is developed, and the computing power of the SW26010 processor is fully utilized via the scratch pad memory (SPM) access strategy and serpentine register communication. In addition, the framework is implemented and tested by the axial compressor rotor simulation algorithm on a real-world dataset with Sunway many-core processors. The results demonstrate that we can achieve a speedup of up to 8.2×, compared to the original ported version, which only uses management processing elements (MPEs), as well as a 1.3× speedup compared to an Intel Xeon E5 processor. The proposed framework is useful for the optimization of fluid mechanical algorithm programs on computing platforms with a heterogeneous many-core architecture.