Nguyen Thanh Binh

This chapter aims to provide an overview of deep learning methods for video classification and tagging. Videos contain not only spatial but also temporal information, making it more complex than static image data. Firstly, the chapter will introduce the basic concepts of video classification, video tagging, single-label classification, multi-label classification and challenges of video data. Secondly, the main part of the chapter will review major deep learning architectures used for video understanding, including CNN-based methods, CNN-RNN models, 3D CNNs, two-stream networks, SlowFast networks, video transformers, and self-supervised learning and discuss multi-modal learning methods that combine visual, audio, textual, and metadata features for more effective video tagging. Finally, the following evaluation metrics such as accuracy, precision, recall, F1-score, etc. will be introduced in this chapter. Moreover, practical use cases in education, surveillance, intelligent transportation systems, social media, and digital content management will be discussed.

Colon polyp segmentation is important in the early detection of colorectal cancer. Advances in artificial intelligence and computer‐aided diagnosis systems have dramatically improved the capabilities of colonoscopy by enhancing the segmentation and detection of colonic polyps. Accurate segmentation is essential, as it allows for the differentiation of polyps from surrounding tissues and aids in assessing their potential malignancy. This paper proposed a method for segmenting colon polyp images by improving the Recalling U‐Net model based on the original U‐Net and U‐Net++ models. The new model aims to take full advantage of multiscale features by introducing full‐scale skip connections, combining low‐level details with high‐level semantics from full‐scale feature maps, but with fewer parameters. The model can develop deep supervision to learn hierarchical representations from full‐scale aggregated feature maps in order to optimize the combined loss function to enhance organ boundaries. Specifically, the proposed method improves four major changes compared to Recalling U‐Net: separate down sampling process, improved encoding path, upgraded bridge and improved recalling block layers. These changes help improve the learning ability and efficiency of the proposed method, resulting in more accurate segmentation results. The proposed method is tested on the Kvasir‐SEG, CVC‐ClinicDB and CVC‐ColonDB datasets with evaluation metrics such as Dice coefficient, Intersection over Union and accuracy. Experimental results presented that the proposed method achieved an accuracy of 98.992%, 97.628%, and 97.695% on these datasets, respectively, and outperforms the results of several recent methods on the same dataset.

Diabetes causes damage to the blood vessels of all organs in the body. In the ocular structures, an elevation in vascular permeability occurs as a result of impairment to the retinal capillaries, leading to the extravasation of plasma into the retina and subsequently inducing the formation of edema. The extracted bright lesion in the retinal blood vessel can help doctors in diabetic retinopathy. Nevertheless, this issue presents many challenges and hard work. To solve this challenge, this paper proposed a method for detecting anomalies in retinal blood vessels utilizing a U-Net++ based on U-Net combined machine learning tradition to aid in the evaluation of diabetic retinopathy. The method consists of five periods: initial filtering of the green channel, separation of blood vessels and retinal mask, separation of the bright lesion’s candidates mask, splitting the bright lesion edges, regenerating and detecting abnormal bright lesions. To increase the accuracy, the proposed method is based on the feature of single-color image initialization. It allows the method to select the objects marked with the make image from among those presented in the mask image. Taking advantage of these features, the proposed method is able to select the marked image of the regeneration as the bright lesion edge image. The proposed method achieves an accuracy of approximately 97.8% and 97.9% with the STARE and DIARETDB1 datasets, respectively. The proposed method outperforms existing techniques in the aforementioned datasets.

Traffic congestion is becoming an increasingly serious and challenging issue in major urban areas. This problem not only causes a waste of time and increased fuel consumption but also contributes to environmental pollution and deterioration of residents’ quality of life. In this study, a new method of predicting the average speed reported by traffic sensors across the city was proposed. In this method, we make the most of two core models: Graph Convolutional Networks and Long Short-Term Memory. The YOLO model is used to analyze images and video during data collection. By leveraging Graphe Convolution Networks ability to capture spatial information, Long Short-Term Memory capacity to model temporal dynamics, and YOLO’s strength in visual object detection, our integrated framework enhances the accuracy of traffic flow predictions at specific locations and time intervals. This comprehensive approach aims to support real-world applications such as adaptive traffic light control, traffic planning support, and congestion alerts. The proposed method outperforms other methods on the Caltrans PeMS dataset.

Neural networks overcome drawbacks of vision tasks by becoming convolutional in a wide range of layers. The salient map is affected by multilevels of strong pixels (superpixels) in global images and that is dependent on the hard threshold for their dividing. Deep neural networks have been established for saliency prediction of segmentation because the feature extraction must be suited to the input data. The convolutional neural network (CNN) also endures conflict between spatial pattern and a likeness of salient objects. Semantic segmentation is one of the approaches to continue classification based on these features. Therefore, upgrading the extraction process can be of use in saliency. In this work, we optimize DeepLab based on an atrous convolutional and a conditional random field (CRF) with a bottleneck in the semantic segmentation method, which serves for classification. The backbone of deep feature extraction is atrous convolution and the bottleneck based on CRF for hybrid saliency in the encoder-decoder system. The classification results are compared with some approaches for saliency prediction of recent deeper methods in an ISIC 2017 dataset. The results give better values not only for saliency prediction for segmentation but also for training and testing for classification.

HLA (Human Leukocyte Antigen) gene is located on chromosome 6’s short arm, including several loci. HLA genotyping is necessary for transplantation, especially in hematopoietic stem cell transplantation. A cross-sectional study to determine the frequency of HLA-A, HLA-B, and HLA-DRB1 alleles of 122 patients admitted to the National Children’s Hospital for Hematopoietic stem cell transplantation from the year between 2010 and 2020. The patients’ HLA genotyping was performed by PCR-SSP method using Micro SSPTM Generic HLA Class I and II DNA Typing Tray (One Lambda). The results showed that the HLA-A locus has 12 alleles of which the highest frequency alleles are A*11 (24.18%), A*02 (23.36%), A*33 (20.90%), and A*24 (14.34%). HLA-B locus has 24 alleles in which the highest frequency alleles are B*15 (24.59%), B*58 (12.30%), B*07 (9.02%), and B*46 (8.20%). HLA-C locus has ten alleles of which the highest frequency alleles are C*03 (24.59%), C*07 (22.12%), and C*04 (13.52%). HLA-DRB1 locus has 13 alleles of which the highest frequency alleles are DRB1*12 (23.77%), DRB1*03 (11.07%), DRB1*15 (10.25%), and DRB1*09 & DRB1*07 (8.20%). There are 1984 haplotypes combined from 4 loci HLA-A, HLA-B, HLA-C, and HLA-DRB1, of which the highest frequency haplotypes are A*11/B*15/C*08/DRB1*12 (1.21%); A*33/B*58/C*03/DRB1*03 (1.06%); A*11/B*15/C*04/DRB1*12 (0.76%); A*11/B*15/C*04/DRB1*12 (0.71%), and A*11/B*15/C*03/DRB1*12 (0.71%). The study provides the frequencies of alleles and haplotypes of HLA genotypes in Hematopoietic Stem Cell Transplantation. This is very important information for choosing stem cell donors and establishing the stem cell bank.