Privacy Leakage via Unrestricted Motion-Position Sensors in the Age of Virtual Reality: A Study of Snooping Typed Input on Virtual Keyboards
Authors:
- 1 May 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 3382-3398
Abstract
Virtual Reality (VR) has gained popularity in numerous fields, including gaming, social interactions, shopping, and education. In this paper, we conduct a comprehensive study to assess the trustworthiness of the embedded sensors on VR, which embed various forms of sensitive data that may put users’ privacy at risk. We find that accessing most on-board sensors (e.g., motion, position, and button sensors) on VR SDKs/APIs, such as OpenVR, Oculus Platform, and WebXR, requires no security permission, exposing a huge attack surface for an adversary to steal the user’s privacy. We validate this vulnerability through developing malware programs and malicious websites and specifically explore to what extent it exposes the user’s information in the context of keystroke snooping. To examine its actual threat in practice, the adversary in the considered attack model doesn’t possess any labeled data from the user nor knowledge about the user’s VR settings. Extensive experiments, involving two mainstream VR systems and four keyboards with different typing mechanisms, demonstrate that our proof-of-concept attack can recognize the user’s virtual typing with over 89.7% accuracy. The attack can recover the user’s passwords with up to 84.9% recognition accuracy if three attempts are allowed and achieve an average of 87.1% word recognition rate for paragraph inference. We hope this study will help the community gain awareness of the vulnerability in the sensor management of current VR systems and provide insights to facilitate the future design of more comprehensive and restricted sensor access control mechanisms.Keywords
This publication has 31 references indexed in Scilit:
- Preventing shoulder surfing using randomized augmented reality keyboards2017
- MoLe2015
- Snooping Keystrokes with mm-level Audio Ranging on a Single Phone2015
- Design and Analysis of Shoulder Surfing Resistant PIN Based Authentication Mechanisms on Google Glass2015
- Android permissions2012
- Tapprints2012
- ACCessory2012
- Keyboard acoustic emanations revisitedACM Transactions on Information and System Security, 2009
- A least square fit programNuclear Instruments and Methods, 1969
- IEEE Recommended Practice for Speech Quality MeasurementsIEEE Transactions on Audio and Electroacoustics, 1969