An IoT-based System of Converting Handwritten Text to Editable Format via Gesture Recognition
Open Access
- 11 October 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 142-147
Abstract
Despite the prevalence of online learning or e-learning, the evolution of traditional classrooms has never stopped. Electronic classrooms, where classrooms equipped with projectors, computers, and sound systems, have become dominant in the last decade. The next step of evolution for electronic classrooms is smart classrooms. One of the most popular features of an electronic classroom is capturing video/photos of lecture content and extracting handwriting for note-taking. Various techniques have been implemented in order to extract handwriting from the video/photos of the lecture. However, there are still deficiencies in the existing techniques that prevent us from turning an electronic classroom into a smart classroom. This paper presents a real-time Internet of Things (loT)-based system to convert handwritten text into editable format by implementing Hand Gesture Recognition (HGR) with Raspberry Pi. Handwritten images are converted into editable format by using OpenCV and machine learning algorithms. For text conversion, the recognition of uppercase and lowercase alphabets, numbers, special characters, mathematical symbols, equations, graphs, and figures is included with the recognition of words, lines, blocks, and paragraphs. With the help of Raspberry Pi and IoT technologies, students can access the editable format of lecture notes via a desktop application, which helps students to edit and share notes and images according to their necessity. Implementation details and comprehensive evaluations of the system are summarized in the paper.Keywords
Funding Information
- NSF (CNS-2244450)
This publication has 13 references indexed in Scilit:
- An integrative review of the cognitive costs and benefits of note-takingEducational Research Review, 2017
- Internet of Things (IoT)-based Learning Framework to Facilitate STEM Undergraduate Education2017
- Robust handwriting extraction and lecture video summarizationMultimedia Tools and Applications, 2016
- Using speech recognition for real-time captioning and lecture transcription in the classroomIEEE Transactions on Learning Technologies, 2013
- Lecture content classification tool2012
- Automatic lecture transcription by exploiting presentation slide information for language model adaptationInternational Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2008
- Video Scene Segmentation Using the State Recognition of Blackboard for Blended Learning2007
- Inclusion of deaf students in computer science classes using real-time speech transcriptionACM SIGCSE Bulletin, 2007
- Content Extraction and Summarization of Instructional Videos2006
- Cognitive effort during note takingApplied Cognitive Psychology, 2004