Mary Lou Soffa
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Mary Lou Soffa has a total of 12 co-authors. Here are the top co-authors by number of shared publications:
- Matthew B. Dwyer
University of Virginia
3 shared publications - Swaroopa Dola
Computer Engineering, University of Virginia, Charlottesville, Virginia, USA
3 shared publications Atif M. Memon
Dept. of Computer Science, University of Pittsburgh, Pittsburgh, PA
1 shared publicationJack W. Davidson
University of Virginia, Charlottesville, Virginia
1 shared publication- Jason Mars
Jaseci Laboratories, University of Michigan, Ann Arbor, MI, USA
1 shared publication - Kevin Skadron
University of Virginia
1 shared publication
Latest Publications
- Open Access
- 12 April 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-13
CIT4DNN: Generating Diverse and Rare Inputs for Neural Networks Using Latent Space Combinatorial Testing
Deep neural networks (DNN) are being used in a wide range of applications including safety-critical systems. Several DNN test generation approaches have been proposed to generate fault-revealing test inputs. However, the existing test generation approaches do not systematically cover the input data distribution to test DNNs with diverse inputs, and none of the approaches investigate the relationship between rare inputs and faults. We propose cit4dnn, an automated black-box approach to generate DNN test sets that are feature-diverse and that comprise rare inputs. cit4dnn constructs diverse test sets by applying combinatorial interaction testing to the latent space of generative models and formulates constraints over the geometry of the latent space to generate rare and fault-revealing test inputs. Evaluation on a range of datasets and models shows that cit4dnn generated tests are more feature diverse than the state-of-the-art, and can target rare fault-revealing testing inputs more effectively than existing methods.
- 12 April 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-13
Open AccessCIT4DNN: Generating Diverse and Rare Inputs for Neural Networks Using Latent Space Combinatorial Testing
Deep neural networks (DNN) are being used in a wide range of applications including safety-critical systems. Several DNN test generation approaches have been proposed to generate fault-revealing test inputs. However, the existing test generation approaches do not systematically cover the input data distribution to test DNNs with diverse inputs, and none of the approaches investigate the relationship between rare inputs and faults. We propose cit4dnn, an automated black-box approach to generate DNN test sets that are feature-diverse and that comprise rare inputs. cit4dnn constructs diverse test sets by applying combinatorial interaction testing to the latent space of generative models and formulates constraints over the geometry of the latent space to generate rare and fault-revealing test inputs. Evaluation on a range of datasets and models shows that cit4dnn generated tests are more feature diverse than the state-of-the-art, and can target rare fault-revealing testing inputs more effectively than existing methods.
- Open Access
- 26 April 2023
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 32 (3) , 1-48
Input Distribution Coverage: Measuring Feature Interaction Adequacy in Neural Network Testing
Testingdeep neural networks (DNNs)has garnered great interest in the recent years due to their use in many applications. Black-box test adequacy measures are useful for guiding the testing process in covering the input domain. However, the absence of input specifications makes it challenging to apply black-box test adequacy measures in DNN testing. TheInput Distribution Coverage (IDC)framework addresses this challenge by using a variational autoencoder to learn a low dimensional latent representation of the input distribution, and then using that latent space as a coverage domain for testing. IDC applies combinatorial interaction testing on a partitioning of the latent space to measure test adequacy. Empirical evaluation demonstrates that IDC is cost-effective, capable of detecting feature diversity in test inputs, and more sensitive than prior work to test inputs generated using different DNN test generation methods. The findings demonstrate that IDC overcomes several limitations of white-box DNN coverage approaches by discounting coverage from unrealistic inputs and enabling the calculation of test adequacy metrics that capture the feature diversity present in the input space of DNNs.
- 26 April 2023
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 32 (3) , 1-48
Open AccessInput Distribution Coverage: Measuring Feature Interaction Adequacy in Neural Network Testing
Testingdeep neural networks (DNNs)has garnered great interest in the recent years due to their use in many applications. Black-box test adequacy measures are useful for guiding the testing process in covering the input domain. However, the absence of input specifications makes it challenging to apply black-box test adequacy measures in DNN testing. TheInput Distribution Coverage (IDC)framework addresses this challenge by using a variational autoencoder to learn a low dimensional latent representation of the input distribution, and then using that latent space as a coverage domain for testing. IDC applies combinatorial interaction testing on a partitioning of the latent space to measure test adequacy. Empirical evaluation demonstrates that IDC is cost-effective, capable of detecting feature diversity in test inputs, and more sensitive than prior work to test inputs generated using different DNN test generation methods. The findings demonstrate that IDC overcomes several limitations of white-box DNN coverage approaches by discounting coverage from unrealistic inputs and enabling the calculation of test adequacy metrics that capture the feature diversity present in the input space of DNNs.
- 1 May 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 205-206
Artifact: Distribution-Aware Testing of Neural Networks Using Generative Models
The artifact used for the experimental evaluation of Distribution-Aware Testing of Neural Networks Using Generative Models is publicly available on GitHub and it is reusable. The artifact consists of python scripts, trained deep neural network model files and data required for running the experiments. It is also provided as a VirtualBox VM image for reproducing the paper results. Users should be familiar with using VirtualBox software and Linux platform to reproduce or reuse the artifact.
- 1 May 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 205-206
Artifact: Distribution-Aware Testing of Neural Networks Using Generative Models
The artifact used for the experimental evaluation of Distribution-Aware Testing of Neural Networks Using Generative Models is publicly available on GitHub and it is reusable. The artifact consists of python scripts, trained deep neural network model files and data required for running the experiments. It is also provided as a VirtualBox VM image for reproducing the paper results. Users should be familiar with using VirtualBox software and Linux platform to reproduce or reuse the artifact.
- Open Access
- 1 December 2013
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Architecture and Code Optimization
- Vol. 10 (4) , 1-25
ReSense
- 1 December 2013
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Architecture and Code Optimization
- Vol. 10 (4) , 1-25
Open AccessReSense
- Open Access
- 3 December 2011
- Published by Association for Computing Machinery (ACM)
- p. 248-259
Bubble-Up
- 3 December 2011
- Published by Association for Computing Machinery (ACM)
- p. 248-259
Open AccessBubble-Up
- Open Access
- 1 September 2001
- Published by Association for Computing Machinery (ACM)
- Vol. 26 (5) , 256-267
Coverage criteria for GUI testing
- 1 September 2001
- Published by Association for Computing Machinery (ACM)
- Vol. 26 (5) , 256-267
Open AccessCoverage criteria for GUI testing