Agricultural Machinery Virtual Assembly System Using Dynamic Gesture Recognitive Interaction Based on a CNN and LSTM Network
Open Access
- 25 November 2021
- journal article
- research article
- Published by Wiley in Mathematical Problems in Engineering
- Vol. 2021, 1-16
Abstract
The agricultural machinery experiment is restricted by the crop production season. Missing the crop growth cycle will extend the machine development period. The use of virtual reality technology to complete preassembly and preliminary experiments can reduce the loss caused by this problem. To improve the intelligence and stability of virtual assembly, this paper proposed a more stable dynamic gesture cognition framework: the TCP/IP protocol constituted the network communication terminal, the leap motion-based vision system constituted the gesture data collection terminal, and the CNN-LSTM network constituted the dynamic gesture recognition classification terminal. The dynamic gesture recognition framework and the harvester virtual assembly platform formed a virtual assembly system to achieve gesture interaction. Through experimental analysis, the improved CNN-LSTM network had a small volume and could quickly establish a stable and accurate gesture recognition model with an average accuracy of 98.0% (±0.894). The assembly efficiency of the virtual assembly system with the framework was improved by approximately 15%. The results showed that the accuracy and stability of this model met the requirements, the corresponding assembly parts were robust in the virtual simulation environment of the whole machine, and the harvesting behaviour in the virtual reality scene was close to the real scene. The virtual assembly system under this framework provided technical support for unmanned farms and virtual experiments on agricultural machinery.Keywords
Funding Information
- National Basic Research Program of China (2017YFD0700100, 2019A050510035)
- International Cooperation in Science and Technology by Department of Science and Technology of Guangdong Province (2017YFD0700100, 2019A050510035)
This publication has 15 references indexed in Scilit:
- The Value of Non-Referential Gestures: A Systematic Review of Their Cognitive and Linguistic Effects in Children’s Language DevelopmentChildren, 2021
- YOLOv3-Litchi Detection Method of Densely Distributed Litchi in Large Vision ScenesMathematical Problems in Engineering, 2021
- Recognition of Hand Gesture Image Using Deep Convolutional Neural NetworkInternational Journal of Image and Graphics, 2021
- EMG hand gesture classification using handcrafted and deep featuresBiomedical Signal Processing and Control, 2021
- Gesture Recognition–Based Smart Training Assistant System for Construction Worker Earplug-Wearing TrainingJournal of Construction Engineering and Management, 2020
- A novel hybrid bidirectional unidirectional LSTM network for dynamic hand gesture recognition with Leap MotionEntertainment Computing, 2020
- Augmented reality-based solar system for e-magazine with 3-D audio effectInternational Journal of Simulation and Process Modelling, 2020
- Concept design in virtual reality of a forestry trailer using a QFD-TRIZ based approachTURKISH JOURNAL OF AGRICULTURE AND FORESTRY, 2013
- Modeling and real-time simulation architectures for virtual prototyping of off-road vehiclesVirtual Reality, 2010
- Cooperated operation of plural hand-robots for automatic harvest systemMathematics and Computers in Simulation, 1996