Brian P. Bailey
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Brian P. Bailey has a total of 48 co-authors. Here are the top co-authors by number of shared publications:
- Wenxuan Wendy Shi
California State Polytechnic University, Pomona, Pomona, CA, USA
3 shared publications - Yifan Song
University of Illinois Urbana-Champaign, Urbana, IL, USA
3 shared publications - Ha-Kyung Kong
Rochester Institute of Technology
2 shared publications - Hyo Jin Do
IBM Research, Cambridge, MA, USA
2 shared publications - Jaewook Lee
Paul G. Allen School of Computer Science & Engineering, University of Washington, Seattle, USA
2 shared publications - Karrie Karahalios
MIT Media Lab, Massachusetts Institute of Technology, Cambridge, MA, USA
2 shared publications
Latest Publications
- Open Access
- 13 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-7
Breaking the Curse of Knowledge: Toward Personalized Jargon Support in Online Meetings
- 13 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-7
Open AccessBreaking the Curse of Knowledge: Toward Personalized Jargon Support in Online Meetings
- Open Access
- 17 February 2026
- Published by Association for Computing Machinery (ACM)
- p. 1517-1518
RIPEL: A Data-Augmented Peer Evaluation System for Assessing Teamwork
- 17 February 2026
- Published by Association for Computing Machinery (ACM)
- p. 1517-1518
Open AccessRIPEL: A Data-Augmented Peer Evaluation System for Assessing Teamwork
- Open Access
- 17 February 2026
- Published by Association for Computing Machinery (ACM)
- p. 992-998
From Data to Action: Empowering Students to Assess and Improve Teamwork with Cross-Tool Log Data
- 17 February 2026
- Published by Association for Computing Machinery (ACM)
- p. 992-998
Open AccessFrom Data to Action: Empowering Students to Assess and Improve Teamwork with Cross-Tool Log Data
- Open Access
- 28 September 2023
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Human-Computer Interaction
- Vol. 7 (CSCW2) , 1-24
Composing Team Compositions: An Examination of Instructors' Current Algorithmic Team Formation Practices
Instructors using algorithmic team formation tools must decide which criteria (e.g., skills, demographics, etc.) to use to group students into teams based on their teamwork goals, and have many possible sources from which to draw these configurations (e.g., the literature, other faculty, their students, etc.). However, tools offer considerable flexibility and selecting ineffective configurations can lead to teams that do not collaborate successfully. Due to such tools' relative novelty, there is currently little knowledge of how instructors choose which of these sources to utilize, how they relate different criteria to their goals for the planned teamwork, or how they determine if their configuration or the generated teams are successful. To close this gap, we conducted a survey (N=77) and interview (N=21) study of instructors using CATME Team-Maker and other criteria-based processes to investigate instructors' goals and decisions when using team formation tools. The results showed that instructors prioritized students learning to work with diverse teammates and performed "sanity checks" on their formation approach's output to ensure that the generated teams would support this goal, especially focusing on criteria like gender and race. However, they sometimes struggled to relate their educational goals to specific settings in the tool. In general, they also did not solicit any input from students when configuring the tool, despite acknowledging that this information might be useful. By opening the "black box" of the algorithm to students, more learner-centered approaches to forming teams could therefore be a promising way to provide more support to instructors configuring algorithmic tools while at the same time supporting student agency and learning about teamwork.
- 28 September 2023
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Human-Computer Interaction
- Vol. 7 (CSCW2) , 1-24
Open AccessComposing Team Compositions: An Examination of Instructors' Current Algorithmic Team Formation Practices
Instructors using algorithmic team formation tools must decide which criteria (e.g., skills, demographics, etc.) to use to group students into teams based on their teamwork goals, and have many possible sources from which to draw these configurations (e.g., the literature, other faculty, their students, etc.). However, tools offer considerable flexibility and selecting ineffective configurations can lead to teams that do not collaborate successfully. Due to such tools' relative novelty, there is currently little knowledge of how instructors choose which of these sources to utilize, how they relate different criteria to their goals for the planned teamwork, or how they determine if their configuration or the generated teams are successful. To close this gap, we conducted a survey (N=77) and interview (N=21) study of instructors using CATME Team-Maker and other criteria-based processes to investigate instructors' goals and decisions when using team formation tools. The results showed that instructors prioritized students learning to work with diverse teammates and performed "sanity checks" on their formation approach's output to ensure that the generated teams would support this goal, especially focusing on criteria like gender and race. However, they sometimes struggled to relate their educational goals to specific settings in the tool. In general, they also did not solicit any input from students when configuring the tool, despite acknowledging that this information might be useful. By opening the "black box" of the algorithm to students, more learner-centered approaches to forming teams could therefore be a promising way to provide more support to instructors configuring algorithmic tools while at the same time supporting student agency and learning about teamwork.
- Open Access
- 28 September 2023
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Human-Computer Interaction
- Vol. 7 (CSCW2) , 1-26
Inform, Explain, or Control: Techniques to Adjust End-User Performance Expectations for a Conversational Agent Facilitating Group Chat Discussions
A conversational agent (CA) effectively facilitates online group discussions at scale. However, users may have expectations about how well the CA would perform that do not match with the actual performance, compromising technology acceptance. We built a facilitator CA that detects a member who has low contribution during a synchronous group chat discussion and asks the person to participate more. We designed three techniques to set end-user expectations about how accurately the CA identifies an under-contributing member: 1)information: explicitly communicating the accuracy of the detection algorithm, 2)explanation: providing an overview of the algorithm and the data used for the detection, and 3)adjustment: enabling users to gain a feeling of control over the algorithm. We conducted an online experiment with 163 crowdworkers in which each group completed a collaborative decision-making task and experienced one of the techniques. Through surveys and interviews, we found that the explanation technique was the most effective strategy overall as it reduced user embarrassment, increased the perceived intelligence of the CA, and helped users better understand the detection algorithm. In contrast, the information technique reduced members' contributions and the adjustment technique led to a more negative perceived discussion experience. We also discovered that the interactions with other team members diluted the effects of the techniques on users' performance expectations and acceptance of the CA. We discuss implications for better designing expectation-setting techniques for AI-team collaboration such as ways to improve collaborative decision outcomes and quality of contributions.
- 28 September 2023
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Human-Computer Interaction
- Vol. 7 (CSCW2) , 1-26
Open AccessInform, Explain, or Control: Techniques to Adjust End-User Performance Expectations for a Conversational Agent Facilitating Group Chat Discussions
A conversational agent (CA) effectively facilitates online group discussions at scale. However, users may have expectations about how well the CA would perform that do not match with the actual performance, compromising technology acceptance. We built a facilitator CA that detects a member who has low contribution during a synchronous group chat discussion and asks the person to participate more. We designed three techniques to set end-user expectations about how accurately the CA identifies an under-contributing member: 1)information: explicitly communicating the accuracy of the detection algorithm, 2)explanation: providing an overview of the algorithm and the data used for the detection, and 3)adjustment: enabling users to gain a feeling of control over the algorithm. We conducted an online experiment with 163 crowdworkers in which each group completed a collaborative decision-making task and experienced one of the techniques. Through surveys and interviews, we found that the explanation technique was the most effective strategy overall as it reduced user embarrassment, increased the perceived intelligence of the CA, and helped users better understand the detection algorithm. In contrast, the information technique reduced members' contributions and the adjustment technique led to a more negative perceived discussion experience. We also discovered that the interactions with other team members diluted the effects of the techniques on users' performance expectations and acceptance of the CA. We discuss implications for better designing expectation-setting techniques for AI-team collaboration such as ways to improve collaborative decision outcomes and quality of contributions.
- Open Access
- 10 June 2023
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Computer-Human Interaction
- Vol. 30 (3) , 1-30
Visualizing Topics and Opinions Helps Students Interpret Large Collections of Peer Feedback for Creative Projects
We deployed a feedback visualization tool to learn how students used the tool for interpreting feedback from peers and teaching assistants. The tool visualizes the topic and opinion structure in a collection of feedback and provides interaction for reviewing providers’ backgrounds. A total of 18 teams engaged with the tool to interpret feedback for course projects. We surveyed students (N = 69) to learn about their sensemaking goals, use of the tool to accomplish those goals, and perceptions of specific features. We interviewed students (N = 12) and TAs (N = 2) to assess the tool’s impact on students’ review processes and course instruction. Students discovered valuable feedback, assessed project quality, and justified design decisions to teammates by exploring specific icon patterns in the visualization. The interviews revealed that students mimicked strategies implemented in the tool when reviewing new feedback without the tool. Students found the benefits of the visualization outweighed the cost of labeling feedback.
- 10 June 2023
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Computer-Human Interaction
- Vol. 30 (3) , 1-30
Open AccessVisualizing Topics and Opinions Helps Students Interpret Large Collections of Peer Feedback for Creative Projects
We deployed a feedback visualization tool to learn how students used the tool for interpreting feedback from peers and teaching assistants. The tool visualizes the topic and opinion structure in a collection of feedback and provides interaction for reviewing providers’ backgrounds. A total of 18 teams engaged with the tool to interpret feedback for course projects. We surveyed students (N = 69) to learn about their sensemaking goals, use of the tool to accomplish those goals, and perceptions of specific features. We interviewed students (N = 12) and TAs (N = 2) to assess the tool’s impact on students’ review processes and course instruction. Students discovered valuable feedback, assessed project quality, and justified design decisions to teammates by exploring specific icon patterns in the visualization. The interviews revealed that students mimicked strategies implemented in the tool when reviewing new feedback without the tool. Students found the benefits of the visualization outweighed the cost of labeling feedback.
- Open Access
- 14 April 2023
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Human-Computer Interaction
- Vol. 7 (CSCW1) , 1-23
To Err is AI: Imperfect Interventions and Repair in a Conversational Agent Facilitating Group Chat Discussions
Conversational agents (CAs) can analyze online conversations using natural language techniques and effectively facilitate group discussions by sending supervisory messages. However, if a CA makes imperfect interventions, users may stop trusting the CA and discontinue using it. In this study, we demonstrate how inaccurate interventions of a CA and a conversational repair strategy can influence user acceptance of the CA, members' participation in the discussion, perceived discussion experience between the members, and group performance. We built a CA that encourages the participation of members with low contributions in an online chat discussion in which a small group (3-6 members) performs a decision-making task. Two types of errors can occur when detecting under-contributing members: 1) false-positive (FP) errors happen when the CA falsely identifies a member as under-contributing and 2) false-negative (FN) errors occur when the CA misses detecting an under-contributing member. We designed a conversational repair strategy that gives users a chance to contest the detection results and the agent sends a correctional message if an error is detected. Through an online study with 175 participants, we found that participants who received FN error messages reported higher acceptance of the CA and better discussion experience, but participated less compared to those who received FP error messages. The conversational repair strategy moderated the effect of errors such as improving the perceived discussion experience of participants who received FP error messages. Based on our findings, we offer design implications for which model should be selected by practitioners between high precision (i.e., fewer FP errors) and high recall (i.e., fewer FN errors) models depending on the desired effects. When frequent FP errors are expected, we suggest using the conversational repair strategy to improve the perceived discussion experience.
- 14 April 2023
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Human-Computer Interaction
- Vol. 7 (CSCW1) , 1-23
Open AccessTo Err is AI: Imperfect Interventions and Repair in a Conversational Agent Facilitating Group Chat Discussions
Conversational agents (CAs) can analyze online conversations using natural language techniques and effectively facilitate group discussions by sending supervisory messages. However, if a CA makes imperfect interventions, users may stop trusting the CA and discontinue using it. In this study, we demonstrate how inaccurate interventions of a CA and a conversational repair strategy can influence user acceptance of the CA, members' participation in the discussion, perceived discussion experience between the members, and group performance. We built a CA that encourages the participation of members with low contributions in an online chat discussion in which a small group (3-6 members) performs a decision-making task. Two types of errors can occur when detecting under-contributing members: 1) false-positive (FP) errors happen when the CA falsely identifies a member as under-contributing and 2) false-negative (FN) errors occur when the CA misses detecting an under-contributing member. We designed a conversational repair strategy that gives users a chance to contest the detection results and the agent sends a correctional message if an error is detected. Through an online study with 175 participants, we found that participants who received FN error messages reported higher acceptance of the CA and better discussion experience, but participated less compared to those who received FP error messages. The conversational repair strategy moderated the effect of errors such as improving the perceived discussion experience of participants who received FP error messages. Based on our findings, we offer design implications for which model should be selected by practitioners between high precision (i.e., fewer FP errors) and high recall (i.e., fewer FN errors) models depending on the desired effects. When frequent FP errors are expected, we suggest using the conversational repair strategy to improve the perceived discussion experience.
- Open Access
- 14 April 2023
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Human-Computer Interaction
- Vol. 7 (CSCW1) , 1-39
The Value of Activity Traces in Peer Evaluations: An Experimental Study
Peer evaluations are a well-established tool for evaluating individual and team performance in collaborative contexts, but are susceptible to social and cognitive biases. Current peer evaluation tools have also yet to address the unique opportunities that online collaborative technologies provide for addressing these biases. In this work, we explore the potential of one such opportunity for peer evaluations: data traces automatically generated by collaborative tools, which we refer to as "activity traces". We conduct a between-subjects experiment with 101 students and MTurk workers, investigating the effects of reviewing activity traces on peer evaluations of team members in an online collaborative task. Our findings show that the usage of activity traces led participants to make more and greater revisions to their evaluations compared to a control condition. These revisions also increased the consistency and participants' perceived accuracy of the evaluations that they received. Our findings demonstrate the value of activity traces as an approach for performing more reliable and objective peer evaluations of teamwork. Based on our findings as well as qualitative analysis of free-form responses in our study, we also identify and discuss key considerations and design recommendations for incorporating activity traces into real-world peer evaluation systems.
- 14 April 2023
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Human-Computer Interaction
- Vol. 7 (CSCW1) , 1-39
Open AccessThe Value of Activity Traces in Peer Evaluations: An Experimental Study
Peer evaluations are a well-established tool for evaluating individual and team performance in collaborative contexts, but are susceptible to social and cognitive biases. Current peer evaluation tools have also yet to address the unique opportunities that online collaborative technologies provide for addressing these biases. In this work, we explore the potential of one such opportunity for peer evaluations: data traces automatically generated by collaborative tools, which we refer to as "activity traces". We conduct a between-subjects experiment with 101 students and MTurk workers, investigating the effects of reviewing activity traces on peer evaluations of team members in an online collaborative task. Our findings show that the usage of activity traces led participants to make more and greater revisions to their evaluations compared to a control condition. These revisions also increased the consistency and participants' perceived accuracy of the evaluations that they received. Our findings demonstrate the value of activity traces as an approach for performing more reliable and objective peer evaluations of teamwork. Based on our findings as well as qualitative analysis of free-form responses in our study, we also identify and discuss key considerations and design recommendations for incorporating activity traces into real-world peer evaluation systems.
- Open Access
- 17 October 2019
- Published by Association for Computing Machinery (ACM)
- p. 763-771
A Testbed for Fun and Effective Features in Spatial Skill Training Games
Certain digital games hold promise as interventions for training spatial skills, a subset of cognitive skills essential for success in STEM fields. However, not much is known about what features of these games are relevant to training spatial skills. Without such knowledge, it is impossible to design spatial skill training games with confidence that they will be effective. In addition, existing games for spatial skill training are not designed to appeal to those who stand the most to benefit from them: students with low spatial skill. In this work, we present a new and improved version of Homeworld Bound, a game designed as a testbed for studying spatially relevant game features. Our updated version is grounded in spatial skill theory and designed to appeal to low spatial skill university students. We present our future plans for analyzing the training effectiveness and player experience of this new game, Homeworld Bound: Redux.
- 17 October 2019
- Published by Association for Computing Machinery (ACM)
- p. 763-771
Open AccessA Testbed for Fun and Effective Features in Spatial Skill Training Games
Certain digital games hold promise as interventions for training spatial skills, a subset of cognitive skills essential for success in STEM fields. However, not much is known about what features of these games are relevant to training spatial skills. Without such knowledge, it is impossible to design spatial skill training games with confidence that they will be effective. In addition, existing games for spatial skill training are not designed to appeal to those who stand the most to benefit from them: students with low spatial skill. In this work, we present a new and improved version of Homeworld Bound, a game designed as a testbed for studying spatially relevant game features. Our updated version is grounded in spatial skill theory and designed to appeal to low spatial skill university students. We present our future plans for analyzing the training effectiveness and player experience of this new game, Homeworld Bound: Redux.
- Open Access
- 4 June 2016
- Published by Association for Computing Machinery (ACM)
- p. 773-784
Social Network, Web Forum, or Task Market?
Increasingly, designers seek feedback on their designs from crowd platforms such as social networks, Web forums, and paid task markets which demand different amounts of social capital, financial resources, and time. Yet it is unknown how the choice of crowd platform affects feedback generation. We conducted an online study where designers created initial designs and revised the designs based on crowd feedback. We measured the quantity, quality, and content of the feedback received at two iterations and from crowds driven by social status, enjoyment, and financial gain. Our results show, for example, that task markets yield more suggestions, online forums provide more process feedback, and social networks give the most suggestions without payment. We contribute an emergent framework for crowd feedback selection, opportunities for enhancing feedback services, and an experimental platform that researchers can adapt to reduce the burden of conducting online studies of design feedback.
- 4 June 2016
- Published by Association for Computing Machinery (ACM)
- p. 773-784
Open AccessSocial Network, Web Forum, or Task Market?
Increasingly, designers seek feedback on their designs from crowd platforms such as social networks, Web forums, and paid task markets which demand different amounts of social capital, financial resources, and time. Yet it is unknown how the choice of crowd platform affects feedback generation. We conducted an online study where designers created initial designs and revised the designs based on crowd feedback. We measured the quantity, quality, and content of the feedback received at two iterations and from crowds driven by social status, enjoyment, and financial gain. Our results show, for example, that task markets yield more suggestions, online forums provide more process feedback, and social networks give the most suggestions without payment. We contribute an emergent framework for crowd feedback selection, opportunities for enhancing feedback services, and an experimental platform that researchers can adapt to reduce the burden of conducting online studies of design feedback.