CryptoAPI-Bench: A Comprehensive Benchmark on Java Cryptographic API Misuses
Authors:
- 1 September 2019
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 49-61
Abstract
Several studies showed that misuses of cryptographic APIs are common in real-world code (e.g., Apache projects and Android apps). There exist several open-sourced and commercial security tools that automatically screen Java programs to detect misuses. In order to compare their accuracy and security guarantees, we develop a comprehensive benchmark named CryptoAPI-Bench. CryptoAPI-Bench consists of 171 unit test cases that cover basic cases, as well as complex cases, including interprocedural, field sensitive, multiple class test cases, and path sensitive data flow of misuse cases. The benchmark also includes correct cases for testing false positive rates. We evaluate CryptoAPI-Bench on four tools, namely, SpotBugs, CryptoGuard, CrySL, and Coverity and present their performance and comparative analysis. Our benchmark is useful for advancing state-of-the-art solutions in the space of misuse detection.Keywords
This publication has 24 references indexed in Scilit:
- Collusive Data Leak and More2017
- The ManyBugs and IntroClass Benchmarks for Automated Repair of C ProgramsIEEE Transactions on Software Engineering, 2015
- SMV-HUNTER: Large Scale, Automated Detection of SSL/TLS Man-in-the-Middle Vulnerabilities in Android Apps2014
- An empirical study of cryptographic misuse in android applications2013
- Why eve and mallory love android2012
- The most dangerous code in the world2012
- Towards standardized benchmarks for Dynamic Software Updating systems2012
- The DaCapo benchmarks2006
- Towards a framework and a benchmark for testing tools for multi‐threaded programsConcurrency and Computation: Practice and Experience, 2006
- Benchmark and framework for encouraging research on multi-threaded testing tools2004