IFIZZ: Deep-State and Efficient Fault-Scenario Generation to Test IoT Firmware
Authors:
Open Access
- 1 November 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 805-816
Abstract
IoT devices are abnormally prone to diverse errors due to harsh environments and limited computational capabilities. As a result, correct error handling is critical in IoT. Implementing correct error handling is non-trivial, thus requiring extensive testing such as fuzzing. However, existing fuzzing cannot effectively test IoT error-handling code. First, errors typically represent corner cases, thus are hard to trigger. Second, testing error-handling code would frequently crash the execution, which prevents fuzzing from testing following deep error paths.In this paper, we propose IFIZZ, a new bug detection system specifically designed for testing error-handling code in Linux-based IoT firmware. IFIZZ first employs an automated binary-based approach to identify realistic runtime errors by analyzing errors and error conditions in closed-source IoT firmware. Then, IFIZZ employs state-aware and bounded error generation to reach deep error paths effectively. We implement and evaluate IFIZZ on 10 popular IoT firmware. The results show that IFIZZ can find many bugs hidden in deep error paths. Specifically, IFIZZ finds 109 critical bugs, 63 of which are even in widely used IoT libraries. IFIZZ also features high code coverage and efficiency, and covers 67.3% more error paths than normal execution. Meanwhile, the depth of error handling covered by IFIZZ is 7.3 times deeper than that covered by the state-of-the-art method. Furthermore, IFIZZ has been practically adopted and deployed in a worldwide leading IoT company. We will open-source IFIZZ to facilitate further research in this area.Keywords
Funding Information
- National Key Research and Development Program of China
This publication has 33 references indexed in Scilit:
- Mining and checking paired functions in device drivers using characteristic fault injectionInformation and Software Technology, 2016
- PIE2015
- Automatic fault injection for driver robustness testing2015
- Firmalice - Automatic Detection of Authentication Bypass Vulnerabilities in Binary Firmware2015
- Do Injected Faults Cause Real Failures? A Case Study of Linux2014
- Prospect2014
- Being SMART about failures2012
- Amplifying tests to validate exception handling code2012
- Robustness Testing of the Windows DDK2007
- Testing of java web services for robustness2004