Andrew Smart
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Andrew Smart has a total of 65 co-authors. Here are the top co-authors by number of shared publications:
- Renee Shelby
Google Research, San Francisco, California, USA
4 shared publications - Ajung Moon
McGill University, Montreal, Canada
3 shared publications - Negar Rostamzadeh
Google Research, Canada
3 shared publications - Shalaleh Rismani
McGill University, Montreal, Canada
3 shared publications - Joshua A. Kroll
Naval Postgraduate School, Monterey, California, United States of America
2 shared publications - Lameck Mbangula Amugongo
Biostatistics and Data Sciences Department, Boehringer Ingelheim Pharma GmbH & Co. KG, Biberach an der Riß, Germany, Germany
2 shared publications
Latest Publications
- Open Access
- 6 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 9546-9549
REACH-AI 2026: Community-Informed Evaluation and Benchmarking of Culturally Competent Generative AI
Recent advances in Generative AI (GenAI) have enabled strong capabilities in language understanding, content generation, vision-language reasoning, and multimodal interaction. However, most existing benchmarks emphasize general-purpose reasoning and task performance, offering limited evaluation of whether AI systems can accurately understand, represent, retrieve, and generate culturally grounded knowledge across diverse communities and social contexts. This limitation restricts the reliable deployment of GenAI in high-impact domains such as education, healthcare, and public services. The REACH-AI 2026 Workshop brings together researchers, industry leaders, practitioners, and community stakeholders to advance community-informed approaches for evaluating culturally competent Generative AI. Motivated by the REACH GenAI Consortium, a collaboration between Google Research and twelve Historically Black Colleges and Universities (HBCUs), the workshop focuses on culturally grounded datasets, community-informed taxonomies, evaluation frameworks, benchmark development, and human-centered assessment protocols for measuring cultural competency in AI systems. It aims to foster interdisciplinary dialogue across the CIKM, information retrieval, knowledge management, multimodal AI, and responsible AI communities. Through invited talks, paper presentations, and panel discussions, participants will explore methods for assessing cultural relevance, contextual understanding, representation, and knowledge utilization in GenAI systems. Expected outcomes include a community roadmap, collaborative research opportunities, and recommendations for future datasets, benchmarks, and evaluation standards for culturally competent AI.
- 6 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 9546-9549
Open AccessREACH-AI 2026: Community-Informed Evaluation and Benchmarking of Culturally Competent Generative AI
Recent advances in Generative AI (GenAI) have enabled strong capabilities in language understanding, content generation, vision-language reasoning, and multimodal interaction. However, most existing benchmarks emphasize general-purpose reasoning and task performance, offering limited evaluation of whether AI systems can accurately understand, represent, retrieve, and generate culturally grounded knowledge across diverse communities and social contexts. This limitation restricts the reliable deployment of GenAI in high-impact domains such as education, healthcare, and public services. The REACH-AI 2026 Workshop brings together researchers, industry leaders, practitioners, and community stakeholders to advance community-informed approaches for evaluating culturally competent Generative AI. Motivated by the REACH GenAI Consortium, a collaboration between Google Research and twelve Historically Black Colleges and Universities (HBCUs), the workshop focuses on culturally grounded datasets, community-informed taxonomies, evaluation frameworks, benchmark development, and human-centered assessment protocols for measuring cultural competency in AI systems. It aims to foster interdisciplinary dialogue across the CIKM, information retrieval, knowledge management, multimodal AI, and responsible AI communities. Through invited talks, paper presentations, and panel discussions, participants will explore methods for assessing cultural relevance, contextual understanding, representation, and knowledge utilization in GenAI systems. Expected outcomes include a community roadmap, collaborative research opportunities, and recommendations for future datasets, benchmarks, and evaluation standards for culturally competent AI.
- Open Access
- 13 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-18
How Tech Workers Contend with Hazards of Humanlikeness in Generative AI
- 13 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-18
Open AccessHow Tech Workers Contend with Hazards of Humanlikeness in Generative AI
- 10 March 2026
- Published by MIT Press in Harvard Data Science Review
- Vol. 8 (3)
Unsafe at any AUC: Unlearned Lessons From Sociotechnical Disasters for Responsible AI
- 10 March 2026
- Published by MIT Press in Harvard Data Science Review
- Vol. 8 (3)
Unsafe at any AUC: Unlearned Lessons From Sociotechnical Disasters for Responsible AI
- Open Access
- 23 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 2249-2259
Lost in Machine Translation: The Sociocultural Implications of Language Technologies in Nigeria
- 23 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 2249-2259
Open AccessLost in Machine Translation: The Sociocultural Implications of Language Technologies in Nigeria
- Open Access
- 16 October 2024
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society
- Vol. 7, 141-155
Unsocial Intelligence: An Investigation of the Assumptions of AGI Discourse
Dreams of machines rivaling human intelligence have shaped the field of AI since its inception. Yet, the very meaning of human-level AI or artificial general intelligence (AGI) remains elusive and contested. Definitions of AGI embrace a diverse range of incompatible values and assumptions. Contending with the fractured worldviews of AGI discourse is vital for critiques that pursue different values and futures. To that end, we provide a taxonomy of AGI definitions, laying the ground for examining the key social, political, and ethical assumptions they make. We highlight instances in which these definitions frame AGI or human-level AI as a technical topic and expose the value-laden choices being implicitly made. Drawing on feminist, STS, and social science scholarship on the political and social character of intelligence in both humans and machines, we propose contextual, democratic, and participatory paths to imagining future forms of machine intelligence. The development of future forms of AI must involve explicit attention to the values it encodes, the people it includes or excludes, and a commitment to epistemic justice.
- 16 October 2024
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society
- Vol. 7, 141-155
Open AccessUnsocial Intelligence: An Investigation of the Assumptions of AGI Discourse
Dreams of machines rivaling human intelligence have shaped the field of AI since its inception. Yet, the very meaning of human-level AI or artificial general intelligence (AGI) remains elusive and contested. Definitions of AGI embrace a diverse range of incompatible values and assumptions. Contending with the fractured worldviews of AGI discourse is vital for critiques that pursue different values and futures. To that end, we provide a taxonomy of AGI definitions, laying the ground for examining the key social, political, and ethical assumptions they make. We highlight instances in which these definitions frame AGI or human-level AI as a technical topic and expose the value-laden choices being implicitly made. Drawing on feminist, STS, and social science scholarship on the political and social character of intelligence in both humans and machines, we propose contextual, democratic, and participatory paths to imagining future forms of machine intelligence. The development of future forms of AI must involve explicit attention to the values it encodes, the people it includes or excludes, and a commitment to epistemic justice.
- Open Access
Socially Responsible Data for Large Multilingual Language Models
Large Language Models (LLMs) have rapidly increased in size and apparent capabilities in the last three years, but their training data is largely English text. There is growing interest in multilingual LLMs, and various efforts are striving for models to accommodate languages of communities outside of the Global North, which include many languages that have been historically underrepresented in digital realms. These languages have been coined as "low resource languages" or "long-tail languages", and LLMs performance on these languages is generally poor. While expanding the use of LLMs to more languages may bring many potential benefits, such as assisting cross-community communication and language preservation, great care must be taken to ensure that data collection on these languages is not extractive and that it does not reproduce exploitative practices of the past. Collecting data from languages spoken by previously colonized people, indigenous people, and non-Western languages raises many complex sociopolitical and ethical questions, e.g., around consent, cultural safety, and data sovereignty. Furthermore, linguistic complexity and cultural nuances are often lost in LLMs. This position paper builds on recent scholarship, and our own work, and outlines several relevant social, cultural, and ethical considerations and potential ways to mitigate them through qualitative research, community partnerships, and participatory design approaches. We provide twelve recommendations for consideration when collecting language data on underrepresented language communities outside of the Global North.
Open AccessSocially Responsible Data for Large Multilingual Language Models
Large Language Models (LLMs) have rapidly increased in size and apparent capabilities in the last three years, but their training data is largely English text. There is growing interest in multilingual LLMs, and various efforts are striving for models to accommodate languages of communities outside of the Global North, which include many languages that have been historically underrepresented in digital realms. These languages have been coined as "low resource languages" or "long-tail languages", and LLMs performance on these languages is generally poor. While expanding the use of LLMs to more languages may bring many potential benefits, such as assisting cross-community communication and language preservation, great care must be taken to ensure that data collection on these languages is not extractive and that it does not reproduce exploitative practices of the past. Collecting data from languages spoken by previously colonized people, indigenous people, and non-Western languages raises many complex sociopolitical and ethical questions, e.g., around consent, cultural safety, and data sovereignty. Furthermore, linguistic complexity and cultural nuances are often lost in LLMs. This position paper builds on recent scholarship, and our own work, and outlines several relevant social, cultural, and ethical considerations and potential ways to mitigate them through qualitative research, community partnerships, and participatory design approaches. We provide twelve recommendations for consideration when collecting language data on underrepresented language communities outside of the Global North.
- Open Access
Beyond Model Interpretability: Socio-Structural Explanations in Machine Learning
What is it to interpret the outputs of an opaque machine learning model. One approach is to develop interpretable machine learning techniques. These techniques aim to show how machine learning models function by providing either model centric local or global explanations, which can be based on mechanistic interpretations revealing the inner working mechanisms of models or nonmechanistic approximations showing input feature output data relationships. In this paper, we draw on social philosophy to argue that interpreting machine learning outputs in certain normatively salient domains could require appealing to a third type of explanation that we call sociostructural explanation. The relevance of this explanation type is motivated by the fact that machine learning models are not isolated entities but are embedded within and shaped by social structures. Sociostructural explanations aim to illustrate how social structures contribute to and partially explain the outputs of machine learning models. We demonstrate the importance of sociostructural explanations by examining a racially biased healthcare allocation algorithm. Our proposal highlights the need for transparency beyond model interpretability, understanding the outputs of machine learning systems could require a broader analysis that extends beyond the understanding of the machine learning model itself.
Open AccessBeyond Model Interpretability: Socio-Structural Explanations in Machine Learning
What is it to interpret the outputs of an opaque machine learning model. One approach is to develop interpretable machine learning techniques. These techniques aim to show how machine learning models function by providing either model centric local or global explanations, which can be based on mechanistic interpretations revealing the inner working mechanisms of models or nonmechanistic approximations showing input feature output data relationships. In this paper, we draw on social philosophy to argue that interpreting machine learning outputs in certain normatively salient domains could require appealing to a third type of explanation that we call sociostructural explanation. The relevance of this explanation type is motivated by the fact that machine learning models are not isolated entities but are embedded within and shaped by social structures. Sociostructural explanations aim to illustrate how social structures contribute to and partially explain the outputs of machine learning models. We demonstrate the importance of sociostructural explanations by examining a racially biased healthcare allocation algorithm. Our proposal highlights the need for transparency beyond model interpretability, understanding the outputs of machine learning systems could require a broader analysis that extends beyond the understanding of the machine learning model itself.
- Top Cited Papers
- 5 September 2024
- Published by Springer Nature in AI & SOCIETY
- Vol. 40 (4) , 2045-2053
Beyond model interpretability: socio-structural explanations in machine learning
- 5 September 2024
- Published by Springer Nature in AI & SOCIETY
- Vol. 40 (4) , 2045-2053
Top Cited PapersBeyond model interpretability: socio-structural explanations in machine learning
- Top Cited PapersOpen Access
The reanimation of pseudoscience in machine learning and its ethical repercussions
Top Cited PapersOpen AccessThe reanimation of pseudoscience in machine learning and its ethical repercussions
- Open Access
Assessing LLMs for Moral Value Pluralism
The fields of AI current lacks methods to quantitatively assess and potentially alter the moral values inherent in the output of large language models (LLMs). However, decades of social science research has developed and refined widely-accepted moral value surveys, such as the World Values Survey (WVS), eliciting value judgments from direct questions in various geographies. We have turned those questions into value statements and use NLP to compute to how well popular LLMs are aligned with moral values for various demographics and cultures. While the WVS is accepted as an explicit assessment of values, we lack methods for assessing implicit moral and cultural values in media, e.g., encountered in social media, political rhetoric, narratives, and generated by AI systems such as LLMs that are increasingly present in our daily lives. As we consume online content and utilize LLM outputs, we might ask, which moral values are being implicitly promoted or undercut, or -- in the case of LLMs -- if they are intending to represent a cultural identity, are they doing so consistently? In this paper we utilize a Recognizing Value Resonance (RVR) NLP model to identify WVS values that resonate and conflict with a given passage of output text. We apply RVR to the text generated by LLMs to characterize implicit moral values, allowing us to quantify the moral/cultural distance between LLMs and various demographics that have been surveyed using the WVS. In line with other work we find that LLMs exhibit several Western-centric value biases; they overestimate how conservative people in non-Western countries are, they are less accurate in representing gender for non-Western countries, and portray older populations as having more traditional values. Our results highlight value misalignment and age groups, and a need for social science informed technological solutions addressing value plurality in LLMs.
Open AccessAssessing LLMs for Moral Value Pluralism
The fields of AI current lacks methods to quantitatively assess and potentially alter the moral values inherent in the output of large language models (LLMs). However, decades of social science research has developed and refined widely-accepted moral value surveys, such as the World Values Survey (WVS), eliciting value judgments from direct questions in various geographies. We have turned those questions into value statements and use NLP to compute to how well popular LLMs are aligned with moral values for various demographics and cultures. While the WVS is accepted as an explicit assessment of values, we lack methods for assessing implicit moral and cultural values in media, e.g., encountered in social media, political rhetoric, narratives, and generated by AI systems such as LLMs that are increasingly present in our daily lives. As we consume online content and utilize LLM outputs, we might ask, which moral values are being implicitly promoted or undercut, or -- in the case of LLMs -- if they are intending to represent a cultural identity, are they doing so consistently? In this paper we utilize a Recognizing Value Resonance (RVR) NLP model to identify WVS values that resonate and conflict with a given passage of output text. We apply RVR to the text generated by LLMs to characterize implicit moral values, allowing us to quantify the moral/cultural distance between LLMs and various demographics that have been surveyed using the WVS. In line with other work we find that LLMs exhibit several Western-centric value biases; they overestimate how conservative people in non-Western countries are, they are less accurate in representing gender for non-Western countries, and portray older populations as having more traditional values. Our results highlight value misalignment and age groups, and a need for social science informed technological solutions addressing value plurality in LLMs.