Michael Xieyang Liu
Michael Xieyang Liu has a total of 49 co-authors. Here are the top co-authors by number of shared publications:
- Michael Terry
Google DeepMind, Cambridge, Massachusetts, USA
5 shared publications - Carrie Jun Cai
Google DeepMind, Mountain View, CA, USA
4 shared publications - Alexander J. Fiannaca
Google DeepMind, Seattle, Washington, USA
3 shared publications - Franklin Mingzhe Li
Human-Computer Interaction Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania, USA
3 shared publications - Aniket Kittur
Carnegie Mellon University, Pittsburgh, PA, USA
2 shared publications - Emily Reif
Google research
2 shared publications
Latest Publications
- Open Access
- 23 September 2026
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Human-Computer Interaction
- Vol. 10 (6) , 1-33
AI LEGO: Scaffolding Cross-Functional Collaboration in Industrial Responsible AI Practices during Early Design Stages
Responsible AI (RAI) efforts increasingly emphasize the importance of addressing potential harms early in the AI development lifecycle through social-technical lenses. However, in cross-functional industry teams, this work is often stalled by a persistent coordination challenge: how technical roles hand off technical intent, how teams establish shared structures for collaboration, and how non-technical roles are supported in systematically evaluating harms. Through literature review and a semi-structured interview study with 8 practitioners, we unpack how this challenge manifests—technical design choices are rarely handed off in ways that support meaningful engagement by non-technical roles; collaborative workflows lack shared, visual structures to support mutual understanding; and non-technical practitioners are left without scaffolds for systematic harm evaluation. Existing tools like JIRA or Google Docs, while useful for product tracking, are ill-suited for supporting joint harm identification across roles, often requiring significant extra effort to align understanding. To address this, we developed AI LEGO, a web-based prototype that operationalizes the boundary object theory to support cross-functional AI practitioners in effectively facilitating knowledge handoff and identifying harmful design choices in the early design stages. Technical roles use interactive blocks to draft development plans, while non-technical roles engage with those blocks through stage-specific checklists and LLM-driven persona simulations to surface potential harms. In a study with 18 cross-functional practitioners, AI LEGO increased the volume and likelihood of harms identified compared to baseline worksheets. Participants found that its modular structure and persona prompts made harm identification more accessible, fostering clearer and more collaborative RAI practices in early design.
- 23 September 2026
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Human-Computer Interaction
- Vol. 10 (6) , 1-33
Open AccessAI LEGO: Scaffolding Cross-Functional Collaboration in Industrial Responsible AI Practices during Early Design Stages
Responsible AI (RAI) efforts increasingly emphasize the importance of addressing potential harms early in the AI development lifecycle through social-technical lenses. However, in cross-functional industry teams, this work is often stalled by a persistent coordination challenge: how technical roles hand off technical intent, how teams establish shared structures for collaboration, and how non-technical roles are supported in systematically evaluating harms. Through literature review and a semi-structured interview study with 8 practitioners, we unpack how this challenge manifests—technical design choices are rarely handed off in ways that support meaningful engagement by non-technical roles; collaborative workflows lack shared, visual structures to support mutual understanding; and non-technical practitioners are left without scaffolds for systematic harm evaluation. Existing tools like JIRA or Google Docs, while useful for product tracking, are ill-suited for supporting joint harm identification across roles, often requiring significant extra effort to align understanding. To address this, we developed AI LEGO, a web-based prototype that operationalizes the boundary object theory to support cross-functional AI practitioners in effectively facilitating knowledge handoff and identifying harmful design choices in the early design stages. Technical roles use interactive blocks to draft development plans, while non-technical roles engage with those blocks through stage-specific checklists and LLM-driven persona simulations to surface potential harms. In a study with 18 cross-functional practitioners, AI LEGO increased the volume and likelihood of harms identified compared to baseline worksheets. Participants found that its modular structure and persona prompts made harm identification more accessible, fostering clearer and more collaborative RAI practices in early design.
- Open Access
- 13 June 2026
- Published by Association for Computing Machinery (ACM)
- p. 1188-1204
Compass vs Railway Tracks: Unpacking User Mental Models for Communicating Long-Horizon Work to Humans vs. AI
- 13 June 2026
- Published by Association for Computing Machinery (ACM)
- p. 1188-1204
Open AccessCompass vs Railway Tracks: Unpacking User Mental Models for Communicating Long-Horizon Work to Humans vs. AI
- Open Access
- 13 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-21
ADCanvas: Accessible and Conversational Audio Description Authoring for Blind and Low Vision Creators
- 13 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-21
Open AccessADCanvas: Accessible and Conversational Audio Description Authoring for Blind and Low Vision Creators
- Open Access
- 25 April 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-10
LLM Adoption in Data Curation Workflows: Industry Practices and Insights
- 25 April 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-10
Open AccessLLM Adoption in Data Curation Workflows: Industry Practices and Insights
- Open Access
- 25 April 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-6
Bidirectional Human-AI Alignment: Emerging Challenges and Opportunities
- 25 April 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-6
Open AccessBidirectional Human-AI Alignment: Emerging Challenges and Opportunities
- Open Access
- 24 March 2025
- Published by Association for Computing Machinery (ACM)
- p. 755-770
Gensors: Authoring Personalized Visual Sensors with Multimodal Foundation Models and Reasoning
- 24 March 2025
- Published by Association for Computing Machinery (ACM)
- p. 755-770
Open AccessGensors: Authoring Personalized Visual Sensors with Multimodal Foundation Models and Reasoning
- Open Access
- 11 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-5
Sensemaking: What is it today?
- 11 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-5
Open AccessSensemaking: What is it today?
- Open Access
- 11 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-14
A Contextual Inquiry of People with Vision Impairments in Cooking
Individuals with vision impairments employ a variety of strategies for object identification, such as pans or soy sauce, in the culinary process. In addition, they often rely on contextual details about objects, such as location, orientation, and current status, to autonomously execute cooking activities. To understand how people with vision impairments collect and use the contextual information of objects while cooking, we conducted a contextual inquiry study with 12 participants in their own kitchens. This research aims to analyze object interaction dynamics in culinary practices to enhance assistive vision technologies for visually impaired cooks. We outline eight different types of contextual information and the strategies that blind cooks currently use to access the information while preparing meals. Further, we discuss preferences for communicating contextual information about kitchen objects as well as considerations for the deployment of AI-powered assistive technologies.
- 11 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-14
Open AccessA Contextual Inquiry of People with Vision Impairments in Cooking
Individuals with vision impairments employ a variety of strategies for object identification, such as pans or soy sauce, in the culinary process. In addition, they often rely on contextual details about objects, such as location, orientation, and current status, to autonomously execute cooking activities. To understand how people with vision impairments collect and use the contextual information of objects while cooking, we conducted a contextual inquiry study with 12 participants in their own kitchens. This research aims to analyze object interaction dynamics in culinary practices to enhance assistive vision technologies for visually impaired cooks. We outline eight different types of contextual information and the strategies that blind cooks currently use to access the information while preparing meals. Further, we discuss preferences for communicating contextual information about kitchen objects as well as considerations for the deployment of AI-powered assistive technologies.
- Top Cited PapersOpen Access
- 11 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-26
Selenite: Scaffolding Online Sensemaking with Comprehensive Overviews Elicited from Large Language Models
Sensemaking in unfamiliar domains can be challenging, demanding considerable user effort to compare different options with respect to various criteria. Prior research and our formative study found that people would benefit from reading an overview of an information space upfront, including the criteria others previously found useful. However, existing sensemaking tools struggle with the “cold-start” problem — it not only requires significant input from previous users to generate and share these overviews, but such overviews may also turn out to be biased and incomplete. In this work, we introduce a novel system, Selenite, which leverages Large Language Models (LLMs) as reasoning machines and knowledge retrievers to automatically produce a comprehensive overview of options and criteria to jumpstart users’ sensemaking processes. Subsequently, Selenite also adapts as people use it, helping users find, read, and navigate unfamiliar information in a systematic yet personalized manner. Through three studies, we found that Selenite produced accurate and high-quality overviews reliably, significantly accelerated users’ information processing, and effectively improved their overall comprehension and sensemaking experience.
- 11 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-26
Top Cited PapersOpen AccessSelenite: Scaffolding Online Sensemaking with Comprehensive Overviews Elicited from Large Language Models
Sensemaking in unfamiliar domains can be challenging, demanding considerable user effort to compare different options with respect to various criteria. Prior research and our formative study found that people would benefit from reading an overview of an information space upfront, including the criteria others previously found useful. However, existing sensemaking tools struggle with the “cold-start” problem — it not only requires significant input from previous users to generate and share these overviews, but such overviews may also turn out to be biased and incomplete. In this work, we introduce a novel system, Selenite, which leverages Large Language Models (LLMs) as reasoning machines and knowledge retrievers to automatically produce a comprehensive overview of options and criteria to jumpstart users’ sensemaking processes. Subsequently, Selenite also adapts as people use it, helping users find, read, and navigate unfamiliar information in a systematic yet personalized manner. Through three studies, we found that Selenite produced accurate and high-quality overviews reliably, significantly accelerated users’ information processing, and effectively improved their overall comprehension and sensemaking experience.
- Top Cited PapersOpen Access
- 11 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-9
"We Need Structured Output": Towards User-centered Constraints on Large Language Model Output
Large language models can produce creative and diverse responses. However, to integrate them into current developer workflows, it is essential to constrain their outputs to follow specific formats or standards. In this work, we surveyed 51 experienced industry professionals to understand the range of scenarios and motivations driving the need for output constraints from a user-centered perspective. We identified 134 concrete use cases for constraints at two levels: low-level, which ensures the output adhere to a structured format and an appropriate length, and high-level, which requires the output to follow semantic and stylistic guidelines without hallucination. Critically, applying output constraints could not only streamline the currently repetitive process of developing, testing, and integrating LLM prompts for developers, but also enhance the user experience of LLM-powered features and applications. We conclude with a discussion on user preferences and needs towards articulating intended constraints for LLMs, alongside an initial design for a constraint prototyping tool.
- 11 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-9
Top Cited PapersOpen Access"We Need Structured Output": Towards User-centered Constraints on Large Language Model Output
Large language models can produce creative and diverse responses. However, to integrate them into current developer workflows, it is essential to constrain their outputs to follow specific formats or standards. In this work, we surveyed 51 experienced industry professionals to understand the range of scenarios and motivations driving the need for output constraints from a user-centered perspective. We identified 134 concrete use cases for constraints at two levels: low-level, which ensures the output adhere to a structured format and an appropriate length, and high-level, which requires the output to follow semantic and stylistic guidelines without hallucination. Critically, applying output constraints could not only streamline the currently repetitive process of developing, testing, and integrating LLM prompts for developers, but also enhance the user experience of LLM-powered features and applications. We conclude with a discussion on user preferences and needs towards articulating intended constraints for LLMs, alongside an initial design for a constraint prototyping tool.