NoteLink: A Point-and-Shoot Linking Interface between Students' Handwritten Notebooks and Instructional Videos
- 1 September 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 140-149
Abstract
When learning from instructional videos, students frequently take handwritten notes to improve recall and comprehension. When reviewing their notes, it can be difficult to return to the corresponding part of the video. In this paper, we present NoteLink, a mobile application that allows students to take pictures of their notes to re-find and play relevant videos on their smartphone or tablet. Our study followed four phases. In Phase I, we identified the characteristics of students' notes by analyzing 10 engineering students' handwritten notes taken while watching instructional videos. We found: 1) students' notes are comprised of four content types: text, formula, drawing, and a hybrid of two or more types, 2) at least 75% of the notes, regardless of content type, manifest some degree of verbatim overlap with the corresponding video content, and 3) videos are referenced at three scales of temporal granularity: point, interval, and whole video. In Phase II, we designed a prototype mobile application, NoteLink, that retrieves instructional videos that are similar to students' notes. In Phase III, we ran a usability study with 12 engineering students to evaluate their preferences for the temporal granularity of retrieved videos and how search results are displayed. Students reported a preference for matches at the interval temporal granularity. Interviews with participants suggest that NoteLink-like tools for re-finding instructional videos are useful. In Phase IV, we evaluated the retrieval accuracy of NoteLink using the data collected in Phase I. The overall accuracy was 78%, and 98% for textual notes. We also provide design recommendations for optimizing NoteLink.Keywords
Funding Information
- University of British Columbia
- Natural Sciences and Engineering Research Council of Canada (NSERC)
- Microsoft
This publication has 50 references indexed in Scilit:
- Contextual text/non-text stroke classification in online handwritten notes with conditional random fieldsPattern Recognition, 2014
- Towards effective video annotation: An approach to automatically link notes with video contentComputers & Education, 2010
- Multimedia surrogates for video gisting: Toward combining spoken words and imageryInformation Processing & Management, 2009
- Hypertext fiction reading: haptics and immersionJournal of Research in Reading, 2008
- Personal Information ManagementAnnual Review of Information Science and Technology, 2007
- Distinctive Image Features from Scale-Invariant KeypointsInternational Journal of Computer Vision, 2004
- Remember-Know: A Matter of Confidence.Psychological Review, 2004
- E-notesACM SIGCSE Bulletin, 2003
- Online and off-line handwriting recognition: a comprehensive surveyIEEE Transactions on Pattern Analysis and Machine Intelligence, 2000
- College students' theory of note-taking derived from their perceptions of note-taking.Journal of Educational Psychology, 1994