Hengxin Chen
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Hengxin Chen has a total of 30 co-authors. Here are the top co-authors by number of shared publications:
Bin Fang
College of Computer Science, Chongqing University, Chongqing, China
2 shared publicationsShuang Dong
Chongqing University,Chongqing,China
2 shared publicationsXinrun Chen
College of Computer Science, Chongqing University, Chongqing, China
2 shared publications- Xinrun Chen
Computer Science, Chongqing University,Chongqing,China
2 shared publications Bing Li
Department of Joint, Tianjin Hospital, Tianjin, China
1 shared publicationChao Qin
Department of Joint, Tianjin Hospital, Tianjin, China
1 shared publication
Latest Publications
- Open Access
- 16 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-11
JoystickWord: Word-Level Overlapped Handwriting for Text Entry with Joystick-Based Controllers
- 16 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-11
Open AccessJoystickWord: Word-Level Overlapped Handwriting for Text Entry with Joystick-Based Controllers
- Open Access
- 6 March 2026
- Published by Springer Nature in Journal of NeuroEngineering and Rehabilitation
- Vol. 23 (1)
The feasibility of using deep learning technologies to preliminarily identify patients with advanced knee osteoarthritis via smartphone videos
Knee osteoarthritis (KOA) is a prevalent condition that often leads to a decline in patients’ physical function. Many patients with KOA struggle to d
- 6 March 2026
- Published by Springer Nature in Journal of NeuroEngineering and Rehabilitation
- Vol. 23 (1)
Open AccessThe feasibility of using deep learning technologies to preliminarily identify patients with advanced knee osteoarthritis via smartphone videos
Knee osteoarthritis (KOA) is a prevalent condition that often leads to a decline in patients’ physical function. Many patients with KOA struggle to d
- Open Access
- 9 July 2025
- Published by Springer Nature in Research Square
SmartKB: Morphologically Adaptive Virtual Keyboard in VR Based on Multimodal Character Prediction
In VR environment, the method of text-input by creating virtual surrogates for the keyboard and hands not only overcomes spatial constraints but also enhances users' sense of immersion. However, the input efficiency of virtual keyboards based on virtual surrogates is hindered by the accuracy of hand tracking. Especially in small vision-based tracking devices, hand occlusion leads to a decline in input efficiency. To enhance the input efficiency of virtual keyboards, we propose an optimization approach for virtual keyboards based on multimodal character prediction. This method integrates historical text and hand - motion trajectory information to build a multimodal character-prediction model. Based on this, a probability fusion strategy grounded in dynamic weights is designed. Probability serves as the driving force to update the key shapes, guiding users to input text efficiently. Compared with virtual keyboards in standard form, this method increases the text-input speed of virtual keyboards from 10.99 WPM to 14.08 WPM, and reduces the total error rate from 0.11 to 0.06.
- 9 July 2025
- Published by Springer Nature in Research Square
Open AccessSmartKB: Morphologically Adaptive Virtual Keyboard in VR Based on Multimodal Character Prediction
In VR environment, the method of text-input by creating virtual surrogates for the keyboard and hands not only overcomes spatial constraints but also enhances users' sense of immersion. However, the input efficiency of virtual keyboards based on virtual surrogates is hindered by the accuracy of hand tracking. Especially in small vision-based tracking devices, hand occlusion leads to a decline in input efficiency. To enhance the input efficiency of virtual keyboards, we propose an optimization approach for virtual keyboards based on multimodal character prediction. This method integrates historical text and hand - motion trajectory information to build a multimodal character-prediction model. Based on this, a probability fusion strategy grounded in dynamic weights is designed. Probability serves as the driving force to update the key shapes, guiding users to input text efficiently. Compared with virtual keyboards in standard form, this method increases the text-input speed of virtual keyboards from 10.99 WPM to 14.08 WPM, and reduces the total error rate from 0.11 to 0.06.
- 16 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Haptics
- Vol. 17 (1) , 26-32
Horizontal Plane Haptic Redirection: Realizing Haptic Feedback for the Virtual Inclined Plane in VR
Some interactions in virtual environments need to be operated on inclined planes. If a real inclined plane can be found in the real environment that corresponds exactly to the angle of the virtual inclined plane to provide haptic feedback, the user's immersion can be enhanced. However, it is not easy to find such a real inclined plane in the real environment. We proposed a horizontal plane haptic redirection scheme, where users interacting with virtual inclined planes in virtual environments can obtain haptic feelings by using real horizontal planes that are easily available in the real world for redirection mapping. We also designed an integrated solution to locate the real horizontal plane and for haptic redirection based on the Vive Pro headset. Then we measured the angle and size thresholds for horizontal plane haptic redirection as 20° and 88%, respectively, through a user study. Through experiments, we also found that when the degree of redirection exceeded the threshold, the user's operation efficiency would be significantly reduced. In addition, we compared the horizontal plane haptic redirection scheme with the scheme without redirection and the scheme without haptic feedback to demonstrate the validity and necessity of the redirection scheme proposed in this paper.
- 16 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Haptics
- Vol. 17 (1) , 26-32
Horizontal Plane Haptic Redirection: Realizing Haptic Feedback for the Virtual Inclined Plane in VR
Some interactions in virtual environments need to be operated on inclined planes. If a real inclined plane can be found in the real environment that corresponds exactly to the angle of the virtual inclined plane to provide haptic feedback, the user's immersion can be enhanced. However, it is not easy to find such a real inclined plane in the real environment. We proposed a horizontal plane haptic redirection scheme, where users interacting with virtual inclined planes in virtual environments can obtain haptic feelings by using real horizontal planes that are easily available in the real world for redirection mapping. We also designed an integrated solution to locate the real horizontal plane and for haptic redirection based on the Vive Pro headset. Then we measured the angle and size thresholds for horizontal plane haptic redirection as 20° and 88%, respectively, through a user study. Through experiments, we also found that when the degree of redirection exceeded the threshold, the user's operation efficiency would be significantly reduced. In addition, we compared the horizontal plane haptic redirection scheme with the scheme without redirection and the scheme without haptic feedback to demonstrate the validity and necessity of the redirection scheme proposed in this paper.
- 1 October 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 4, 184-189
Object Detection under Finger Occlusion in AR Geography Assisted Teaching System by Using PCCNet
Augmented reality(AR) plays an important role in geography teaching for creating interactive and immersive experiences. Combining object detection algorithms with AR can identify the specified content quickly and thus overlay digital content to the real world. However, finger occlusion in AR interactions has a bad influence on object detection, which will affect the users' experience. In this paper, we focus on the detection of country regions on a globe, and aim to improve the performance of object detection in practical AR development. Firstly, we propose a geographic region recognition approach based on region missing-completion. Specifically, we design a supplementary algorithm PCCNet to infer the obscured country by utilizing the invariance of relative position between countries. Moreover, to reduce manual annotation and enrich the virtual dataset, we design a scalable automatic annotation system based on the Unreal Engine and construct a virtual globe dataset named DGAR. Finally, we build an AR geography-assisted teaching system to recognize the area pointed and play multi-media materials. Experiment results show that our proposed approach effectively improves the recognition accuracy from 88.5% to 94%. The practical significance and value of the proposed recognition approach have been confirmed based on the positive user experience with the AR system, highlighting its efficacy in real world scenarios.
- 1 October 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 4, 184-189
Object Detection under Finger Occlusion in AR Geography Assisted Teaching System by Using PCCNet
Augmented reality(AR) plays an important role in geography teaching for creating interactive and immersive experiences. Combining object detection algorithms with AR can identify the specified content quickly and thus overlay digital content to the real world. However, finger occlusion in AR interactions has a bad influence on object detection, which will affect the users' experience. In this paper, we focus on the detection of country regions on a globe, and aim to improve the performance of object detection in practical AR development. Firstly, we propose a geographic region recognition approach based on region missing-completion. Specifically, we design a supplementary algorithm PCCNet to infer the obscured country by utilizing the invariance of relative position between countries. Moreover, to reduce manual annotation and enrich the virtual dataset, we design a scalable automatic annotation system based on the Unreal Engine and construct a virtual globe dataset named DGAR. Finally, we build an AR geography-assisted teaching system to recognize the area pointed and play multi-media materials. Experiment results show that our proposed approach effectively improves the recognition accuracy from 88.5% to 94%. The practical significance and value of the proposed recognition approach have been confirmed based on the positive user experience with the AR system, highlighting its efficacy in real world scenarios.
- Open Access
- 18 August 2023
- Published by Springer Nature in Research Square
Quick Porting of Virtual Simulation Experiment Interactions from the Web 3D to VR
Virtual reality (VR) technology is developing rapidly, and many 3D virtual simulation experiments on the web can be quickly ported to the VR environment to enhance user experience. The manner of interacting with a quickly ported two-dimensional user interface (2DUI) is a matter of preference. In this study, we focused on finding a more effective interaction scheme when a web 3D virtual simulation experiment is ported into the VR environment. When the development time is short, ray interaction and touch interaction can be easily and quickly ported while retaining the 2DUI of the original web-side experiment. We selected the architecture optics virtual simulation experiment for porting, divided the interactions into two levels, namely, L1 ray and L2 touch interactions, and recruited 30 volunteers to participate in the experiment in groups. The efficiency and user experience of L1 and L2 were compared by designing interactive tasks and collecting data. Overall, L1 was more efficient than L2. For the four common user interfaces, buttons, sliders, toggles, and dropdowns, L2 was only slightly better at interacting with the dropdown boxes than L1. However, both were slower than the web interface. L2 also received higher workload scores, based on user feedback and workload surveys. We also designed an interaction level, L3, for VR with reference to Worlds in Miniature and compared it with L1/L2/web by using the same interaction tasks. L3 was as efficient as the web. Finally, based on the experimental results, we provide suggestions for quick porting development.
- 18 August 2023
- Published by Springer Nature in Research Square
Open AccessQuick Porting of Virtual Simulation Experiment Interactions from the Web 3D to VR
Virtual reality (VR) technology is developing rapidly, and many 3D virtual simulation experiments on the web can be quickly ported to the VR environment to enhance user experience. The manner of interacting with a quickly ported two-dimensional user interface (2DUI) is a matter of preference. In this study, we focused on finding a more effective interaction scheme when a web 3D virtual simulation experiment is ported into the VR environment. When the development time is short, ray interaction and touch interaction can be easily and quickly ported while retaining the 2DUI of the original web-side experiment. We selected the architecture optics virtual simulation experiment for porting, divided the interactions into two levels, namely, L1 ray and L2 touch interactions, and recruited 30 volunteers to participate in the experiment in groups. The efficiency and user experience of L1 and L2 were compared by designing interactive tasks and collecting data. Overall, L1 was more efficient than L2. For the four common user interfaces, buttons, sliders, toggles, and dropdowns, L2 was only slightly better at interacting with the dropdown boxes than L1. However, both were slower than the web interface. L2 also received higher workload scores, based on user feedback and workload surveys. We also designed an interaction level, L3, for VR with reference to Worlds in Miniature and compared it with L1/L2/web by using the same interaction tasks. L3 was as efficient as the web. Finally, based on the experimental results, we provide suggestions for quick porting development.
- 24 February 2023
- Published by Springer Nature in Virtual Reality
- Vol. 27 (3) , 1797-1812
Action recognition based on multimode fusion for VR online platform
The current popular online communication platforms can convey information only in the form of text, voice, pictures, and other electronic means. The richness and reliability of information is not comparable to traditional face-to-face communication. The use of virtual reality (VR) technology for online communication is a viable alternative to face-to-face communication. In the current VR online communication platform, users are in a virtual world in the form of avatars, which can achieve “face-to-face” communication to a certain extent. However, the actions of the avatar do not follow the user, which makes the communication process less realistic. Decision-makers need to make decisions based on the behavior of VR users, but there are no effective methods for action data collection in VR environments. In our work, three modalities of nine actions from VR users are collected using a virtual reality head-mounted display (VR HMD) built-in sensors, RGB cameras and human pose estimation. Using these data and advanced multimodal fusion action recognition networks, we obtained a high accuracy action recognition model. In addition, we take advantage of the VR HMD to collect 3D position data and design a 2D key point augmentation scheme for VR users. Using the augmented 2D key point data and VR HMD sensor data, we can train action recognition models with high accuracy and strong stability. In data collection and experimental work, we focus our research on classroom scenes, and the results can be extended to other scenes.
- 24 February 2023
- Published by Springer Nature in Virtual Reality
- Vol. 27 (3) , 1797-1812
Action recognition based on multimode fusion for VR online platform
The current popular online communication platforms can convey information only in the form of text, voice, pictures, and other electronic means. The richness and reliability of information is not comparable to traditional face-to-face communication. The use of virtual reality (VR) technology for online communication is a viable alternative to face-to-face communication. In the current VR online communication platform, users are in a virtual world in the form of avatars, which can achieve “face-to-face” communication to a certain extent. However, the actions of the avatar do not follow the user, which makes the communication process less realistic. Decision-makers need to make decisions based on the behavior of VR users, but there are no effective methods for action data collection in VR environments. In our work, three modalities of nine actions from VR users are collected using a virtual reality head-mounted display (VR HMD) built-in sensors, RGB cameras and human pose estimation. Using these data and advanced multimodal fusion action recognition networks, we obtained a high accuracy action recognition model. In addition, we take advantage of the VR HMD to collect 3D position data and design a 2D key point augmentation scheme for VR users. Using the augmented 2D key point data and VR HMD sensor data, we can train action recognition models with high accuracy and strong stability. In data collection and experimental work, we focus our research on classroom scenes, and the results can be extended to other scenes.
- 23 November 2022
- Published by Springer Nature in Virtual Reality
- Vol. 27 (3) , 1717-1732
Emotion recognition using facial expressions in an immersive virtual reality application
- 23 November 2022
- Published by Springer Nature in Virtual Reality
- Vol. 27 (3) , 1717-1732
Emotion recognition using facial expressions in an immersive virtual reality application
- 12 April 2021
- Published by Wiley in Computer Applications in Engineering Education
- Vol. 29 (6) , 1702-1717
A virtual reality experiment system for an introductory computer hardware course
The University Computer Foundation (UCF) course, which is a compulsory theoretical course for freshmen, especially science and engineering department students, lays the foundation for more advanced courses. However, the UCF course covers a wide variety of concepts, including the computer hardware, which are difficult for freshmen to comprehend. In recent years, virtual reality (VR) has become prevalent in many educational settings. To improve the teaching of the UCF course, we developed a VR program about introductory computer hardware and invited freshmen from two universities in China to take part in the experiment as participants. Four aspects of computer contents, namely the evolutionary history of computers, the computer components, the computer assembly, and the workflow of computer hardware, are displayed in the program. During the experiment, students' behavioral data were recorded. Then, the data were analyzed and the reasons were speculated for the differences among groups of various categories. From the experiment, the relationships between the students' behavior and their groups were found, and we demonstrated that our VR program is effective at attracting the students' curiosity and increasing their understanding of computer hardware in the UCF course. The study concludes with suggestions for practitioners and researchers in the field of VR for university education.
- 12 April 2021
- Published by Wiley in Computer Applications in Engineering Education
- Vol. 29 (6) , 1702-1717
A virtual reality experiment system for an introductory computer hardware course
The University Computer Foundation (UCF) course, which is a compulsory theoretical course for freshmen, especially science and engineering department students, lays the foundation for more advanced courses. However, the UCF course covers a wide variety of concepts, including the computer hardware, which are difficult for freshmen to comprehend. In recent years, virtual reality (VR) has become prevalent in many educational settings. To improve the teaching of the UCF course, we developed a VR program about introductory computer hardware and invited freshmen from two universities in China to take part in the experiment as participants. Four aspects of computer contents, namely the evolutionary history of computers, the computer components, the computer assembly, and the workflow of computer hardware, are displayed in the program. During the experiment, students' behavioral data were recorded. Then, the data were analyzed and the reasons were speculated for the differences among groups of various categories. From the experiment, the relationships between the students' behavior and their groups were found, and we demonstrated that our VR program is effective at attracting the students' curiosity and increasing their understanding of computer hardware in the UCF course. The study concludes with suggestions for practitioners and researchers in the field of VR for university education.
- 22 February 2021
- Published by World Scientific Pub Co Pte Ltd in International Journal of Wavelets, Multiresolution and Information Processing
- Vol. 19 (04)
Building pose estimation from the perspective of UAVs based on CNNs
- 22 February 2021
- Published by World Scientific Pub Co Pte Ltd in International Journal of Wavelets, Multiresolution and Information Processing
- Vol. 19 (04)
Building pose estimation from the perspective of UAVs based on CNNs