Xingang Liu

Deep learning has been proved to be one of the most effective method in feature encoding for different intelligent applications such as video-based human action recognition. However, its nonconvex optimization mechanism leads large memory consumption, which hinders its deployment on the smart embedded systems with limited computational resources. To overcome this challenge, we propose a novel deep model compression technique for smart embedded systems, which realizes both the memory size reduction and inference complexity decrease within a small drop of accuracy. First, we propose an improved naive Bayes inference-based channel parameter pruning to obtain a sparse model with higher accuracy. Then, to improve the inference efficiency, the improved Tucker decomposition method is proposed, where an improved genetic algorithm is used to optimize the Tucker ranks. Finally, to elevate the effectiveness of our proposed method, extensive experiments are conducted. The experimental results show that our method can achieve the state-of-the-art performance compared with existing methods in terms of accuracy, parameter compression, and floating-point operations reduction.

Human behavior understanding is playing more and more important role in human-centered Industrial Internet of Video Things (IIoVT) system with the deep combination of artificial intelligence and video-based industrial Internet of Things. However, it requires expensively computational resources, including high-performance computing units and large memory, to train a deep computation model with a large number of parameters, which limits its effectiveness and efficiency for IIoVT applications. In this article, a tensor-train mechanism based deep model is presented for video human behavior understanding to meet the requirement of IIoVT applications. It can get competitive performance in accuracy and training efficiency with potentiality for combination of artificial intelligence and prefront IIoVT system. On the one hand, to achieve desirable accuracy, we improved the conventional CNN and adopted the recurrent neural network mechanism to enhance the video representation over time, which takes the correlation between consecutive deep feature into consideration. On the other hand, to enhance the inference capacity between the spatial and temporal features, we carry out the self-critical reinforcement learning mechanism in parameter learning stage. Meanwhile, to further reduce parameter storage size to meet requirement for the deployment of deep neural network and edge device, the tensor-train mechanism is used, which transforms the parameter matrix to a tensor space and carry out tensor decomposition mechanism to decrease the number of parameter generated in parameter training. Finally, we conduct extensive experiments to evaluate our scheme, and the results demonstrate that our method can improve the training efficiency and save the memory space for the deep computation model with better accuracy.

Multimedia Internet-of-Things (IoT) systems have been widely utilized in various computer vision tasks and significantly integrated computer vision and networking capabilities. In these systems, convolutional neural networks (CNNs) perform a preliminary analysis of the collected video or image information in the edge devices. However, the high computational cost and huge storage consumption of the complex CNNs prevent their deployment on mobile-edge devices that have limited computational resource and memory. In this article, we aim to simultaneously accelerate and compress CNNs via a multilevel filter pruning (MFP) algorithm, to alleviate the dependence on the hardware of IoT edge nodes. First, a global pruning sensitivity order is defined, which could guide us to perform preliminary pruning from the perspective of convolutional layers' sensitivity. Then, the functional index of each filter is judged by the image entropy of its output feature map, which contributes to further pruning from the perspective of filter function importance. Finally, the moderate fine tuning is adopted to recover the network capability. The experimental results show that the proposed MFP algorithm could reduce 54.5% floating-point operations and 31.9% graphics memory for VGG-16 on CIFAR-10, and achieve 5.45 × floating-point acceleration and 19.70 × storage reduction for VGG-16 on ImageNet. In the reconstruction phase, the algorithm could recover the network capability much faster than the existing pruning algorithms.

Video surveillance techniques like scene segmentation are playing an increasingly important role in multimedia Internet-of-Things (IoT) systems. However, existing deep learning based methods face challenges in both accuracy and memory when deployed on edge computing devices with limited computing resources. To address these challenges, a tensor-train video scene segmentation scheme that compares the local background information in regional scene boundary boxes in adjacent frames, is proposed. Compared to existing methods, the proposed scheme can achieve competitive performance in both segmentation accuracy and parameter compression rate. In detail,B first, an improved faster Region Convolutional Neural NetworkB (faster-RCNN) model is proposed to recognize and generate a large number of region boxes with foreground and background to achieve boundary boxes. Then the foreground boxes with sparse objects are removed and the rest are considered as optional background boxes used to measure the similarity between two adjacent frames.B Second, to accelerate the training efficiency and reduce memory size, a general and efficient training way using tensor-train decomposition to factor the input-to-hidden weight matrix, is proposed. Finally, experiments are conducted to evaluate the performance of the proposed scheme in terms of the accuracy and model compression. Our results demonstrate that the proposed model can improve the training efficiency and save the memory space for the deep computation model with good accuracy. This work opens the potential for the use of artificial intelligence methods in edge computing devices for multimedia IoT systems.

The emergence of edge computing (EC) and intelligent vision-based driver assistance system is of great significance for the prospective development of Internet of Vehicle (IoV). The additional computation capability and extensive network coverage provides energy-limited smart devices with more op- portunities to enable IoV system for time-sensitive applications. However, the transmission of a large amount of redundant data not only causes data transmission delay but also severely compromises the accuracy of object detection when implement- ed in a vision-based driver assistance system. In this paper, an improved object detection algorithm based on video key- frame for latency reduction on edge IoV system is proposed. It can significantly improve latency reduction performance at the expense of small detection accuracy. In our proposal, we adopt an important coefficient and frame similarity comparison algorithm to filter redundant frames and achieve key frames for object detection. Then an improved Haar-like feature based classification algorithm is used for objection detection under the edge computation model. Finally, a scalable cluster object detection system as a practical EC case is implemented to verify our proposal, and extensive simulations confirm the superiority of the proposed scheme over regular schemes. It can speed up about 84 times with 40% of the similar frames filtered in comparison.