Duc-Phong Nguyen

Skull prediction from the head is a challenging issue toward a cost-effective therapeutic solution for facial disorders. This issue was initially studied in our previous work using full head-to-skull relationship learning. However, the head-skull thickness topology is locally shaped, especially in the face region. Thus, the objective of the present study was to enhance our head-to-skull prediction problem by using local topological features for training and predicting. Head and skull feature points were sampled on 329 head and skull models from computed tomography (CT) images. These feature points were classified into the back and facial topologies. Head-to-skull relations were trained using the partial least square regression (PLSR) models separately in the two topologies. A hyperparameter tuning process was also conducted for selecting optimal parameters for each training model. Thus, a new skull could be generated so that its shape was statistically fitted with the target head. Mean errors of the predicted skulls using the topology-based learning method were better than those using the non-topology-based learning method. After tenfold cross-validation, the mean error was enhanced 36.96% for the skull shapes and 14.17% for the skull models. Mean error in the facial skull region was especially improved with 4.98%. The mean errors were also improved 11.71% and 25.74% in the muscle attachment regions and the back skull regions respectively. Moreover, using the enhanced learning strategy, the errors (mean ± SD) for the best and worst prediction cases are from 1.1994 ± 1.1225 mm (median: 0.9036, coefficient of multiple determination (R²): 0.997274) to 3.6972 ± 2.4118 mm (median: 3.9089, R²: 0.999614) and from 2.0172 ± 2.0454 mm (median: 1.2999, R²: 0.995959) to 4.0227 ± 2.6098 mm (median: 3.9998, R²: 0.998577) for the predicted skull shapes and the predicted skull models respectively. This present study showed that more detailed information on the head-skull shape leads to a better accuracy level for the skull prediction from the head. In particular, local topological features on the back and face regions of interest should be considered toward a better learning strategy for the head-to-skull prediction problem. In perspective, this enhanced learning strategy was used to update our developed clinical decision support system for facial disorders. Furthermore, a new class of learning methods, called geometric deep learning will be studied. Graphical abstract

Facial expression recognition plays an essential role in human conversation and human–computer interaction. Previous research studies have recognized facial expressions mainly based on 2D image processing requiring sensitive feature engineering and conventional machine learning approaches. The purpose of the present study was to recognize facial expressions by applying a new class of deep learning called geometric deep learning directly on 3D point cloud data. Two databases (Bosphorus and SIAT-3DFE) were used. The Bosphorus database includes sixty-five subjects with seven basic expressions (i.e., anger, disgust, fearness, happiness, sadness, surprise, and neutral). The SIAT-3DFE database has 150 subjects and 4 basic facial expressions (neutral, happiness, sadness, and surprise). First, preprocessing procedures such as face center cropping, data augmentation, and point cloud denoising were applied on 3D face scans. Then, a geometric deep learning model called PointNet++ was applied. A hyperparameter tuning process was performed to find the optimal model parameters. Finally, the developed model was evaluated using the recognition rate and confusion matrix. The facial expression recognition accuracy on the Bosphorus database was 69.01% for 7 expressions and could reach 85.85% when recognizing five specific expressions (anger, disgust, happiness, surprise, and neutral). The recognition rate was 78.70% with the SIAT-3DFE database. The present study suggested that 3D point cloud could be directly processed for facial expression recognition by using geometric deep learning approach. In perspectives, the developed model will be applied for facial palsy patients to guide and optimize the functional rehabilitation program. Graphical abstract