Abstract
The conventional method of evaluating human actions and activities requires integrating complex hardware and interpretation systems. This article proposes using a hybrid segmentation-classification system to recognize human sports action automatically based on still frames. This study tested the proposed framework on a public dataset containing images of humans performing three athletic actions: dancing, performing martial arts, and playing net sports. This framework combined a two-stage U-Net and EfficientNet-B0, and GoogleNet, and is optimized using particle swarm optimization (PSO) and grid search for cascaded segmentation and classification problems. Results indicated that PSO improves segmentation and classification accuracies by 5 % and 60 %, respectively, compared to the conventional grid search. The PSO segmentation results showed good agreement between the predicted mask and its ground truth, with overlapping scores of 0.85-0.9. The consecutive classification experiments revealed a slight superiority in the performance of EfficientNet-B0 with an accuracy of 0.86-0.88 over GoogleNet (~0.82-0.84), which also showed a lower convergence efficiency. While no significant difference was observed in the recall scores of EfficientNet between the weighted and unweighted loss methods, GoogleNet showed a considerable improvement in the true positive rates from 0.6177 to 0.7160 using the weighted loss strategy. Despite using the weighted loss method, there were negligible improvements in the performance of grid search-optimized networks. The poor classification results suggest low model generalization ability using manual tuning. This study concluded that the PSO-optimized cascaded segmentation-classification framework could potentially leverage advancements in human movement evaluation and rehabilitation assessment for sports applications. This system can be improved by adopting larger datasets or collaborative machine learning training to enhance model convergence for practical rehabilitation applications to assess users’ physical and fitness performance.