GIST : Generated Inputs Sets Transferability in Deep Learning
Open Access
- 30 November 2024
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 33 (8) , 1-38
Abstract
To foster the verifiability and testability of deep neural networks (DNN), an increasing number of methods for test case generation techniques are being developed. When confronted with testing DNN models, the user can apply any existing test generation technique. However, it needs to do so for each technique and each DNN model under test, which can be expensive. Therefore, a paradigm shift could benefit this testing process: rather than regenerating the test set independently for each DNN model under test, we could transfer from existing DNN models. This article introduces Generated Inputs Sets Transferability (GIST), a novel approach for the efficient transfer of test sets. Given a property selected by a user (e.g., neurons covered, faults), GIST enables the selection of good test sets from the point of view of this property among available test sets. This allows the user to recover similar properties on the transferred test sets as he would have obtained by generating the test set from scratch with a test cases generation technique. Experimental results show that GIST can select effective test sets for the given property to transfer. Moreover, GIST scales better than reapplying test case generation techniques from scratch on DNN models under test.Keywords
All Related Versions
This publication has 54 references indexed in Scilit:
- Faster or Slower? Performance Mystery of Python Idioms Unveiled with Empirical Evidence2023
- When and Why Test Generators for Deep Learning Produce Invalid Inputs: an Empirical Study2023
- How to certify machine learning based safety-critical systems? A systematic literature reviewAutomated Software Engineering, 2022
- On testing machine learning programsJournal of Systems and Software, 2020
- Test Transfer Across Mobile Apps Through Semantic Mapping2019
- Feature Space Perturbations Yield More Transferable Adversarial Examples2019
- Guiding Deep Learning System Testing Using Surprise Adequacy2019
- Structural Coverage Criteria for Neural Networks Could Be Misleading2019
- Deep Learning for Audio Signal ProcessingIEEE Journal of Selected Topics in Signal Processing, 2019
- Silhouettes: A graphical aid to the interpretation and validation of cluster analysisJournal of Computational and Applied Mathematics, 1987