Yu-Cheng Lin

Recent advancements in Artificial Intelligence (AI) and Machine Learning (ML) have achieved remarkable success in natural language processing, computer vision, and reinforcement learning. However, when applied to human behavior understanding and motion prediction, most deep learning architectures remain black-box models—lacking interpretability in how perceptual cues and motor actions interact to form human decision-making patterns. To address this limitation, this study proposes an Explainable Multimodal Event Knowledge Graph (E²-KG) framework that integrates eye-tracking data and OpenPose-based joint angles to construct a semantic representation of cognitive–motor coupling in dynamic tasks. Instead of treating perception and motion as separate streams, E²-KG embeds them within a unified perception–action manifold, where local energy gradients automatically detect event transitions (fixation, preparation, release, follow-through). These events are organized into a knowledge graph structure, enabling interpretable reasoning about causal links between visual attention, body movement, and task outcomes. A perception–action manifold model capturing the joint evolution of gaze and motion through local energy variations. A multimodal event extraction pipeline for synchronized eye–pose data. An event knowledge graph that supports cause–effect reasoning. Validation through a throwing-distance prediction task, demonstrating higher interpretability and transparency than data-driven baselines. E²-KG thus bridges manifold learning and knowledge-graph reasoning, offering a transparent, cognitively grounded approach for explainable human performance modeling—applicable to domains such as human factors, adaptive training, and human–robot collaboration.

Emotion detection is a fundamental component in the field of Affective Computing. Proper recognition of emotions can be useful in improving the interaction between humans and machines, for instance, with regard to designing effective user interfaces. This study aims to understand the relationship between emotion and pupil dilation. The Tobii Pro X3-120 eye tracker was used to collect pupillary responses from 30 participants exposed to content designed to evoke specific emotions. Six different video scenarios were selected and presented to participants, whose pupillary responses were measured while watching the material. In total, 16 data features (8 features per eye) were extracted from the pupillary response distribution during content exposure. Through logistical regression, a maximum of 76% classification accuracy was obtained through the measurement of pupillary response in predicting emotions classified as fear, anger, or surprise. Further research is required to precisely calculate pupil size variations in relation to emotionally evocative input in affective computing applications.

Most existing methods for forecasting the productivity of a factory cannot estimate the range of productivity reliably, especially when future conditions are distinct from those in the past. To address this issue, a fuzzified feedforward neural network (FFNN) approach is proposed in this study. The FFNN approach improves the forecasting precision after generating accurate fuzzy productivity forecasts. In addition, the acceptable range of a fuzzy productivity forecast is specified, based on which the sum of the memberships of actual values is maximized. In this way, the range of productivity can be precisely estimated. After applying the FFNN approach to a real case, the experimental results revealed the superiority of the FFNN approach by improving the forecasting precision, in terms of the hit rate, by 25%. Such an improvement also contributed to a better forecasting accuracy. The superiority of the FFNN approach is in the context that the accuracy of forecasting productivity is optimized only after the range of productivity has been precisely estimated. In contrast, most state-of-the-art methods focus on optimizing the forecasting accuracy, but may be ineffective without information about the range of productivity when future conditions are distinct from the past.

A fuzzy-neural fluctuation smoothing rule is proposed in this study to improve the performance of scheduling jobs with various priorities in a semiconductor manufacturing factory. The fuzzy-neural fluctuation smoothing rule is modified from the well-known fluctuation smoothing rule by improving the accuracy of estimating the remaining cycle time of a job, which is done by applying Chen's fuzzy-neural approach with multiple buckets. To evaluate the effectiveness of the proposed methodology, production simulation is also applied in this study. According to experimental results, incorporating a more accurate remaining cycle time estimation mechanism did improve the scheduling performance especially in reducing the average cycle times. Besides, the fuzzy-neural fluctuation smoothing rule was also shown to be a Pareto optimal solution for scheduling jobs with various priorities in a semiconductor manufacturing factory.