GLGrace A. Lewis

Grace A. Lewis

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Grace A. Lewis has a total of 226 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Quality Model for Machine Learning Components

    by Grace A. Lewis, Rachel Brower-Sinning, Robert Edman, Ipek Ozkaya, Sebastian Echeverria, Alex Derr, Collin Beaudoin, Katherine R. Maffey

    Despite increased adoption and advances in machine learning (ML), studies show that many ML prototypes do not reach the production stage and that testing is still largely limited to testing model properties, such as model performance, without considering requirements derived from the system it will be a part of. In traditional software development, quality models such as ISO 25010 provide a structured framework to assess software quality, define quality requirements, and provide a common language for communication with stakeholders. A newer standard, ISO 25059, defines a quality model for AI systems. However, a problem with this standard is that it combines system attributes with ML component attributes, which is not helpful for a model developer, as many system attributes cannot be assessed at the component level. We present a quality model for ML components that serves as a guide for requirements elicitation and negotiation and provides a common vocabulary for ML component developers and system stakeholders to agree on and define system-derived requirements and focus their testing efforts accordingly. The quality model was validated through a survey in which participants agreed with its relevance and value. The quality model has been successfully integrated into MLTE, an open-source tool for ML component test and evaluation.


  • Open Access

    Quality Model for Machine Learning Components

    by Grace A. Lewis, Rachel Brower-Sinning, Robert Edman, Ipek Ozkaya, Sebastián Echeverría, Alex Derr, Collin Beaudoin, Katherine R. Maffey

    Despite increased adoption and advances in machine learning (ML), there are studies showing that many ML prototypes do not reach the production stage and that testing is still largely limited to testing model properties, such as model performance, without considering requirements derived from the system it will be a part of, such as throughput, resource consumption, or robustness. This limited view of testing leads to failures in model integration, deployment, and operations. In traditional software development, quality models such as ISO 25010 provide a widely used structured framework to assess software quality, define quality requirements, and provide a common language for communication with stakeholders. A newer standard, ISO 25059, defines a more specific quality model for AI systems. However, a problem with this standard is that it combines system attributes with ML component attributes, which is not helpful for a model developer, as many system attributes cannot be assessed at the component level. In this paper, we present a quality model for ML components that serves as a guide for requirements elicitation and negotiation and provides a common vocabulary for ML component developers and system stakeholders to agree on and define system-derived requirements and focus their testing efforts accordingly. The quality model was validated through a survey in which the participants agreed with its relevance and value. The quality model has been successfully integrated into an open-source tool for ML component testing and evaluation demonstrating its practical application.

  • Sharing the Journey

    by Grace Lewis, Hausi Müller

    IEEE Computer Society 2026 president Grace Lewis discusses the journey ahead with Hausi Müller. Her goals include improving connections with student members transitioning to their professional lives and what encouragement and mentorship have on the leaders of tomorrow.

  • Open Access

    What Is Wrong with My Model? Identifying Systematic Problems with Semantic Data Slicing

    by Chenyang Yang, Yining Hong, Grace Lewis, Tongshuang Wu, Christian Kästner

    Machine learning models make mistakes, yet sometimes it is difficult to identify the systematic problems behind the mistakes. Practitioners engage in various activities, including error analysis, testing, auditing, and red-teaming, to form hypotheses of what can go (or has gone) wrong with their models. To validate these hypotheses, practitioners employ data slicing to identify relevant examples. However, traditional data slicing is limited by available features and programmatic slicing functions. In this work, we propose SemSlicer, a framework that supports semantic data slicing, which identifies a semantically coherent slice, without the need for existing features. SemSlicer uses Large Language Models to annotate datasets and generate slices from any user-defined slicing criteria. We show that SemSlicer generates accurate slices with low cost, allows flexible trade-offs between different design dimensions, reliably identifies under-performing data slices, and helps practitioners identify useful data slices that reflect systematic problems.


  • Bringing Software Engineering Discipline to the Development of AI-Enabled Systems

    by Miroslaw Staron, Silvia Abrahão, Grace Lewis, Henry Muccini, Chetan Honnenahalli

    Engineering AI Software systems is starting to evolve from the pure development of machine learning (ML) models to a more structured discipline that treats ML components as part of much larger software systems. As such, more structured principles are required for their development, such as established design principles, established development models, and safeguards for deployed ML models. This column focuses on papers presented at the Third International Conference on AI Engineering—Software Engineering for AI (CAIN 2024). The selected papers reflect the current development of the field of AI systems engineering and AI software development, taking it to the next level of maturity. Feedback or suggestions are welcome. In addition, if you try or adopt any of the practices included in the column, please send us and the authors of the paper(s) a note about your experiences.


  • Open Access

    A Synthesis of Green Architectural Tactics for ML-Enabled Systems

    by Heli Järvenpää, Patricia Lago, Justus Bogner, Grace Lewis, Henry Muccini, Ipek Ozkaya

    The rapid adoption of artificial intelligence (AI) and machine learning (ML) has generated growing interest in understanding their environmental impact and the challenges associated with designing environmentally friendly ML-enabled systems. While Green AI research, i.e., research that tries to minimize the energy footprint of AI, is receiving increasing attention, very few concrete guidelines are available on how ML-enabled systems can be designed to be more environmentally sustainable. In this paper, we provide a catalog of 30 green architectural tactics for ML-enabled systems to fill this gap. An architectural tactic is a high-level design technique to improve software quality, in our case environmental sustainability. We derived the tactics from the analysis of 51 peer-reviewed publications that primarily explore Green AI, and validated them using a focus group approach with three experts. The 30 tactics we identified are aimed to serve as an initial reference guide for further exploration into Green AI from a software engineering perspective, and assist in designing sustainable ML-enabled systems. To enhance transparency and facilitate their widespread use and extension, we make the tactics available online in easily consumable formats. Wide-spread adoption of these tactics has the potential to substantially reduce the societal impact of ML-enabled systems regarding their energy and carbon footprint.


  • Open Access

    Computation offloading for ground robotic systems communicating over WiFi – an empirical exploration on performance and energy trade-offs

    by Milica Ðorđević, Michel Albonico, Grace A. Lewis, Ivano Malavolta, Patricia Lago

    Context: Robotic systems are known to perform computation-intensive tasks with limited computational resources and battery life. Such systems might benefit from offloading heavy workloads to the Cloud; however, in some cases, this implies high network traffic that degrades performance and energy consumption.Goal: In this study, we aim at evaluating the impact of different computation offloading strategies on performance and energy consumption in the context of autonomous robots.Method: We conduct two controlled experiments involving a robotic mission based on the Turtlebot3 robot and ROS 1. The mission consists of three tasks that are recurrent in robotics and good candidates for computation offloading in research, namely, SLAM mapping, navigation stack, and object recognition. Each of the tasks is either executed on board or offloaded in a full-factorial experiment design. The obtained measures are then statistically analyzed.Results: The results show that offloading the object recognition task causes a more significant decrease in resource utilization and energy consumption than both SLAM mapping and navigation. However, object recognition affects the volume of network traffic significantly to the extent that it can easily cause network congestion.Conclusions: In the context of our experiments (i.e.,those involving small-scale ground ROS-based mobile robots operating under WiFi networks), offloading object recognition is beneficial in terms of performance and energy consumption. Nevertheless, large network bandwidth needs to be available for object recognition offloading. While the image resolution and frame rate have a significant impact on not only the network traffic but also energy consumption and performance, these parameters need to be carefully set so that the results of this task can be always received in time, which is particularly crucial in real-time systems.


  • Open Access

    Self-Adaptation in Industry: A Survey

    by Danny Weyns, Ilias Gerostathopoulos, Nadeem Abbas, Jesper Andersson, Stefan Biffl, Premek Brada, Tomas Bures, Amleto Di Salle, Matthias Galster, Patricia Lago, et al.

    Computing systems form the backbone of many areas in our society, from manufacturing to traffic control, healthcare, and financial systems. When software plays a vital role in the design, construction, and operation, these systems are referred to as software-intensive systems. Self-adaptation equips a software-intensive system with a feedback loop that either automates tasks that otherwise need to be performed by human operators or deals with uncertain conditions. Such feedback loops have found their way to a variety of practical applications; typical examples are an elastic cloud to adapt computing resources and automated server management to respond quickly to business needs. To gain insight into the motivations for applying self-adaptation in practice, the problems solved using self-adaptation and how these problems are solved, and the difficulties and risks that industry faces in adopting self-adaptation, we performed a large-scale survey. We received 184 valid responses from practitioners spread over 21 countries. Based on the analysis of the survey data, we provide an empirically grounded overview the of state of the practice in the application of self-adaptation. From that, we derive insights for researchers to check their current research with industrial needs, and for practitioners to compare their current practice in applying self-adaptation. These insights also provide opportunities for applying self-adaptation in practice and pave the way for future industry-research collaborations.


  • Open Access

    Data Leakage in Notebooks: Static Detection and Better Processes

    by Chenyang Yang, Rachel A Brower-Sinning, Grace Lewis, Christian Kästner

    Data science pipelines to train and evaluate models with machine learning may contain bugs just like any other code. Leakage between training and test data can lead to overestimating the model’s accuracy during offline evaluations, possibly leading to deployment of low-quality models in production. Such leakage can happen easily by mistake or by following poor practices, but may be tedious and challenging to detect manually. We develop a static analysis approach to detect common forms of data leakage in data science code. Our evaluation shows that our analysis accurately detects data leakage and that such leakage is pervasive among over 100,000 analyzed public notebooks. We discuss how our static analysis approach can help both practitioners and educators, and how leakage prevention can be designed into the development process.


  • Open Access

    IEEE Computer Society

    by William Gropp, Nita Patel, Forrest Shull, Riccardo Mariani, David S. Ebert, Jyotika Athavale, Michela Taufer, André Oboler, Annette Reilly, Christina Schober, et al.