ArmSpy++: Enhanced PIN Inference through Video-based Fine-grained Arm Posture Analysis
Open Access
- 9 November 2024
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Privacy and Security
- Vol. 28 (1) , 1-26
Abstract
As one of the most common ways for user authentication, Personal Identification Number (PIN), due to its simplicity and convenience, has suffered from plenty of side-channel attacks, which pose a severe threat to people’s privacy and property. The success of existing attacks is usually built upon the premise of no occlusion between the attacker and the victim’s hand gesture, but it increases the difficulty of launching the attack and the possibility of exposure. To overcome such limitation, we propose ArmSpy++, an improved video-assisted PIN inference attack built upon our previous research, ArmSpy. Specifically, ArmSpy++ employs new modules to leverage more features like the keystroke-induced elbow bending, wrist speed variation, and the spatial relationship between different arm joints, to correctly detect Keystrokes. ArmSpy++ delves into the perspective relationship and natural typing habits to ensure a high success rate of PIN inference. We also re-designed the inferred PIN pattern coordination mechanism to accurately deduce the PINs. By using a pre-trained HigherHRNet model for posture estimation ArmSpy++ eliminates the necessity of additional training. The extensive experiments demonstrate that ArmSpy++ can achieve over 83.1% average accuracy with 3 attempts and even 92.5% for some victims, indicating the severity of the threat posed by ArmSpy++.Keywords
Funding Information
- National Natural Science Foundation of China (62172080, 62271124)
- Natural Science Foundation of Sichuan Province (2023NSFSC0478, 2023NSFSC0488)
- National Key R&D Program of China (2022YFB3103404)
This publication has 12 references indexed in Scilit:
- UltraPIN: Inferring PIN Entries via Ultrasound2021
- WiHF: Enable User Identified Gesture Recognition with WiFi2020
- Time Reversal Based Robust Gesture Recognition Using Wifi2020
- Keystroke inference using ambient light sensor on wrist-wearables2018
- PPG-based Finger-level Gesture Recognition Leveraging Wearables2018
- Differential audio analysis: a new side-channel attack on PIN padsInternational Journal of Information Security, 2018
- I am a Smartwatch and I can Track my User's Arm2016
- Friend or Foe?2016
- Keyboard acoustic emanations2004
- Smoothing and Differentiation of Data by Simplified Least Squares Procedures.Analytical Chemistry, 1964