RESTful API Automated Test Case Generation
Author:
Preprint
Abstract
Nowadays, web services play a major role in the development of enterprise applications. Many such applications are now developed using a service-oriented architecture (SOA), where microservices is one of its most popular kind. A RESTful web service will provide data via an API over the network using HTTP, possibly interacting with databases and other web services. Testing a RESTful API poses challenges, as inputs/outputs are sequences of HTTP requests/responses to a remote server. Many approaches in the literature do black-box testing, as the tested API is a remote service whose code is not available. In this paper, we consider testing from the point of view of the developers, which do have full access to the code that they are writing. Therefore, we propose a fully automated white-box testing approach, where test cases are automatically generated using an evolutionary algorithm. Tests are rewarded based on code coverage and fault finding metrics. We implemented our technique in a tool called EVOMASTER, which is open-source. Experiments on two open-source, yet non-trivial RESTful services and an industrial one, do show that our novel technique did automatically find 38 real bugs in those applications. However, obtained code coverage is lower than the one achieved by the manually written test suites already existing in those services. Research directions on how to further improve such approach are therefore discussed.Keywords
All Related Versions
- Version 1, 2019-01-06, ArXiv
- Published version: , 9.
This publication has 12 references indexed in Scilit:
- A detailed investigation of the effectiveness of whole test suite generationEmpirical Software Engineering, 2016
- 1600 faults in 100 projects: automatically finding faults while achieving high coverage with EvoSuiteEmpirical Software Engineering, 2013
- Whitening SOA Testing via Event ExposureIEEE Transactions on Software Engineering, 2013
- Parameter tuning or default values? An empirical investigation in search-based software engineeringEmpirical Software Engineering, 2013
- Whole Test Suite GenerationIEEE Transactions on Software Engineering, 2012
- Random Testing: Theoretical Results and Practical ImplicationsIEEE Transactions on Software Engineering, 2011
- Bringing white-box testing to Service Oriented Architectures through a Service Oriented ApproachJournal of Systems and Software, 2011
- A Systematic Review of the Application and Empirical Investigation of Search-Based Test Case GenerationIEEE Transactions on Software Engineering, 2009
- Testing services and service-centric systems: challenges and opportunitiesIT Professional, 2006
- Automated software test data generationIEEE Transactions on Software Engineering, 1990