AAAndrew Anderson

Andrew Anderson

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Andrew Anderson has a total of 53 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    From Verification Burden to Trusted Collaboration: Design Goals for LLM-Assisted Literature Reviews

    by Brenda Nogueira, Werner Geyer, Andrew A. Anderson, Toby Jia-Jun Li, Dongwhi Kim, Nuno Moniz, Nitesh V Chawla

    Although numerous studies have examined how researchers use Large Language Models (LLMs) in academic writing, their concrete implementation, limitations, and design challenges in the literature review process remain underexplored. In this paper, we present findings from a user study with researchers across multiple disciplines that characterize current practices, benefits, and pain points when using LLMs to investigate related work. We identified three recurring gaps: (i) lack of trust in outputs, (ii) a persistent verification burden, and (iii) fragmentation across tools. To address these gaps, we propose six design goals and a high-level framework that operationalizes them through improved visualization of related papers, verification at every step, and human-feedback alignment with generation-guided explanations


  • Open Access

    "Over-the-Hood" AI Inclusivity Bugs and How 3 AI Product Teams Found and Fixed Them

    by Andrew A. Anderson, Fatima A. Moussaoui, Jimena Noa-Guevara, Montaser Hamid, Margaret Burnett

    While much research has shown the presence of AI’s “under-the-hood” biases (e.g., algorithmic, training data, etc.), what about “over-the-hood” inclusivity biases: barriers in user-facing AI products that disproportionately exclude users with certain problem-solving approaches? Recent research has begun to report the existence of such biases—but what do they look like, how prevalent are they, and how can developers find and fix them? To find out, we conducted a field study with 3 AI product teams, to investigate what kinds of AI inclusivity bugs exist uniquely in user-facing AI products, and whether/how AI product teams might harness an existing (non-AI-oriented) inclusive design method to find and fix them. The teams’ work revealed 83 instances of 6 AI inclusivity bug types unique to user-facing AI products, their fixes covering 47 bug instances, and a new GenderMag inclusive design method variant, GenderMag-for-AI, that is especially effective at detecting AI inclusivity bugs when the AI’s output is not necessarily believed.


  • Top Cited Papers
    Open Access

    Inclusive Design of AI’s Explanations: Just for Those Previously Left Out?

    by Montaser Hamid, Fatima A. Moussaoui, Jimena Noa Guevara, Andrew Anderson, Puja Agarwal, Jonathan Dodge, Margaret Burnett

    Motivations . Explainable AI (XAI) systems aim to improve users’ understanding of AI, but XAI research has shown that many XAI explanations serve some users well while failing others. In non-AI systems, software practitioners have used inclusive design approaches to address similar problems, sometimes creating “curb-cut” improvements that benefit both underserved users and everyone else. This raises the possibility that inclusive design approaches can bring similar curb-cut improvements to AI explanations. Objectives . Our objective was to investigate possible curb-cut effects of inclusivity-driven fixes an AI product team made using an inclusive design approach (GenderMag) to improve their XAI prototype. Methods . We ran a between-subject study with 69 participants who had no formal AI background. 34 participants used the original version of the XAI prototype and the rest used the version with the AI team’s inclusivity fixes. We then compared the two groups’ mental model concepts scores and prediction accuracy, and the two prototypes’ inclusivity. Results . Our investigation produced four main results. First, the AI team’s inclusivity fixes were overall effective, resulting in overall better conceptual mental models with the new prototype. Further (second), the AI team’s inclusivity fixes were particularly beneficial to the underserved population’s conceptual mental models—which, together with the first result, constitutes a curb-cut effect. However (third), the inclusivity fixes did not improve participants’ prediction accuracy scores. Instead, it appears to have harmed them overall—a “curb-fence” effect (opposite of a curb-cut effect). Finally (fourth), the AI team’s fixes improved equity, reducing the gender gap by 45%.


  • Open Access

    "Over-the-Hood" AI Inclusivity Bugs and How 3 AI Product Teams Found and Fixed Them

    by Andrew Anderson, Fatima A. Moussaoui, Jimena Noa Guevara, Montaser Hamid, Margaret Burnett

    While much research has shown the presence of AI's "under-the-hood" biases (e.g., algorithmic, training data, etc.), what about "over-the-hood" inclusivity biases: barriers in user-facing AI products that disproportionately exclude users with certain problem-solving approaches? Recent research has begun to report the existence of such biases -- but what do they look like, how prevalent are they, and how can developers find and fix them? To find out, we conducted a field study with 3 AI product teams, to investigate what kinds of AI inclusivity bugs exist uniquely in user-facing AI products, and whether/how AI product teams might harness an existing (non-AI-oriented) inclusive design method to find and fix them. The teams' work resulted in identifying 6 types of AI inclusivity bugs arising 83 times, fixes covering 47 of these bug instances, and a new variation of the GenderMag inclusive design method, GenderMag-for-AI, that is especially effective at detecting certain kinds of AI inclusivity bugs.

  • Open Access

    Measuring SES-related traits relating to technology usage: Two validated surveys

    by Chimdi Chikezie, Pannapat Chanpaisaeng, Puja Agarwal, Sadia Afroz, Bhavika Madhwani, Rudrajit Choudhuri, Andrew Anderson, Prisha Velhal, Patricia Morreale, Christopher Bogart, et al.

    Software producers are now recognizing the importance of improving their products’ suitability for diverse populations, but little attention has been given to measurements to shed light on products’ suitability to individuals below the median s ocio e conomic s tatus (SES)—who, by definition, make up half the population. To enable software practitioners to attend to both lower- and higher-SES individuals, this paper provides two new surveys that together can facilitate measuring how well a software product serves socioeconomically diverse populations. The first survey (SES-Subjective) is who-oriented: it measures who their potential or current users are in terms of their subjective SES (perceptions of their SES). The second survey (SES-Facets) is why-oriented: it collects individuals’ values for an evidence-based set of facet values (individual traits) that (1) statistically differ by SES and (2) affect how an individual works and problem-solves with software products. The surveys’ design goal is worldwide applicability, but as a first step, here we empirically validated both these surveys with deployments at University A and University B (464 and 522 responses, respectively), which showed reliability of both the surveys in a US context. Our results also statistically agree with both ground truth data on respondents’ socioeconomic statuses and with predictions from foundational literature. Finally, we explain how the pair of surveys can be uniquely actionable by software practitioners, such as in requirements gathering, debugging, quality assurance activities, maintenance activities, and fulfilling legal reporting requirements such as those being drafted by various governments for AI-powered software.


  • Open Access

    Towards Personalized and Contextualized Code Explanations

    by Michelle Brachman, Arielle Goldberg, Andrew Anderson, Stephanie Houde, Michael Muller, Justin D. Weisz


  • Open Access

    Measuring User Experience Inclusivity in Human-AI Interaction via Five User Problem-Solving Styles

    by Andrew Anderson, Jimena Noa Guevara, Fatima Moussaoui, Tianyi Li, Mihaela Vorvoreanu, Margaret Burnett

    Motivations : Recent research has emerged on generally how to improve AI products’ human-AI interaction (HAI) user experience (UX), but relatively little is known about HAI-UX inclusivity. For example, what kinds of users are supported, and who are left out? What product changes would make it more inclusive? Objectives : To help fill this gap, we present an approach to measuring what kinds of diverse users an AI product leaves out and how to act upon that knowledge. To bring actionability to the results, the approach focuses on users’ problem-solving diversity. Thus, our specific objectives were (1) to show how the measure can reveal which participants with diverse problem-solving styles were left behind in a set of AI products and (2) to relate participants’ problem-solving diversity to their demographic diversity, specifically gender and age. Methods : We performed 18 experiments, discarding two that failed manipulation checks. Each experiment was a 2 \(\times\) 2 factorial experiment with online participants, comparing two AI products: one deliberately violating 1 of 18 HAI guidelines and the other applying the same guideline. For our first objective, we used our measure to analyze how much each AI product gained/lost HAI-UX inclusivity compared to its counterpart, where inclusivity meant supportiveness to participants with particular problem-solving styles. For our second objective, we analyzed how participants’ problem-solving styles aligned with their gender identities and ages. Results and Implications : Participants’ diverse problem-solving styles revealed six types of inclusivity results: (1) the AI products that followed an HAI guideline were almost always more inclusive across diversity of problem-solving styles than the products that did not follow that guideline—but “who” got most of the inclusivity varied widely by guideline and by problem-solving style; (2) when an AI product had risk implications, four variables’ values varied in tandem: participants’ feelings of control, their (lack of) suspicion, their trust in the product, and their certainty while using the product; (3) the more control an AI product offered users, the more inclusive it was; (4) whether an AI product was learning from “my” data or other people’s affected how inclusive that product was; (5) participants’ problem-solving styles skewed differently by gender and age group; and (6) almost all of the results suggested actions that HAI practitioners could take to improve their products’ inclusivity further. Together, these results suggest that a key to improving the demographic inclusivity of an AI product (e.g., across a wide range of genders, ages) can often be obtained by improving the product’s support of diverse problem-solving styles.


  • Open Access

    Measuring User Experience Inclusivity in Human-AI Interaction via Five User Problem-Solving Styles

    by Andrew Anderson, Jimena Noa Guevara, Fatima Moussaoui, Tianyi Li, Mihaela Vorvoreanu, Margaret Burnett

    Motivations: Recent research has emerged on generally how to improve AI product user experiences, but relatively little is known about an AI product's inclusivity. For example, what kinds of users does it support well, and who does it leave out? And what changes in the product would make it more inclusive? Objectives: Our overall objective is to help fill this gap, investigating what kinds of diverse users an AI product leaves out, and how to act upon that knowledge. To bring actionability to our findings, we focus on users' diversity of problem-solving attributes. Thus, our specific objectives were: (1) to reveal whether participants with diverse problem-solving styles were left behind in a set of AI products; and (2) to relate participants' problem-solving diversity to their demographic diversity, specifically, gender and age. Methods: We performed 18 experiments, discarding two that failed manipulation checks. Each experiment was a 2x2 factorial experiment with online participants. Each experiment compared two AI products: one deliberately violating an HAI guideline and the other applying the guideline. For our first objective, we analyzed how much each AI product gained/lost inclusivity compared to its counterpart, where inclusivity was supportiveness to participants with particular problem-solving styles. For our second objective, we analyzed how participants' problem-solving styles aligned with their demographics, namely their genders and ages. Results & Implications: Participants' diverse problem-solving styles revealed six types of inclusivity results: (1) the AI products that followed an HAI guideline were almost always more inclusive across diversity of problem-solving styles than the products that did not follow that guideline-but the "who" that got most of the inclusivity varied widely by guideline and by problem-solving style...


  • Open Access

    Domino Saliency Metrics: Improving Existing Channel Saliency Metrics with Structural Information

    by Kaveena Persand, Andrew Anderson, David Gregg

    Channel pruning is used to reduce the number of weights in a Convolutional Neural Network (CNN). Channel pruning removes slices of the weight tensor so that the convolution layer remains dense. The removal of these weight slices from a single layer causes mismatching number of feature maps between layers of the network. A simple solution is to force the number of feature map between layers to match through the removal of weight slices from subsequent layers. This additional constraint becomes more apparent in DNNs with branches where multiple channels need to be pruned together to keep the network dense. Popular pruning saliency metrics do not factor in the structural dependencies that arise in DNNs with branches. We propose Domino metrics (built on existing channel saliency metrics) to reflect these structural constraints. We test Domino saliency metrics against the baseline channel saliency metrics on multiple networks with branches. Domino saliency metrics improved pruning rates in most tested networks and up to 25% in AlexNet on CIFAR-10.