Gustavo Kreia Umbelino
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Gustavo Kreia Umbelino has a total of 13 co-authors. Here are the top co-authors by number of shared publications:
- Kristine Lu
Northeastern University, Boston, Massachusetts, USA
4 shared publications - Matthew Easterday
Northwestern University, Evanston, IL, USA
4 shared publications - Francesco Veri
Centre for Democracy Studies Aarau, University of Zurich, Aarau, Switzerland
3 shared publications - April Yi Wang
ETH Zurich, Zurich, Switzerland
2 shared publications - Kiara Chau
ETH Zurich
2 shared publications - Morgan Wu
School of Education & Social Policy, Northwestern University, Evanston, Illinois, USA
2 shared publications
Latest Publications
- Open Access
- 9 September 2026
- Published by National Academy of Sciences in Proceedings of the National Academy of Sciences
- Vol. 123 (37)
Plausible nonsense and deliberative reasoning: Benchmarking LLMs against human judgment
Large Language Models (LLMs) are entering democratic contexts as instruments of governance, where the challenges at hand are ill-structured, marked by ambiguity and contestation. Ill-structured democratic problems demand more than factual precision; they call for intersubjective reasoning: context-sensitive judgments that others can understand and publicly accept. Using the Deliberative Reason Index (DRI), this study evaluates 60 LLMs against human deliberation across nine policy scenarios. Only four models consistently exceed the permutation-based null benchmark for alignment with human patterns of reason-giving. Most models fall short: their reason–preference structures rarely clear this threshold, even though their outputs can still appear coherent and persuasive. Yet outputs can appear reasonable even when this alignment is absent. The observed gap between surface plausibility and deliberative coherence urges caution: deploying LLMs in governance contexts requires prior assessment of their deliberative reasoning capacity, not just their surface outputs.
- 9 September 2026
- Published by National Academy of Sciences in Proceedings of the National Academy of Sciences
- Vol. 123 (37)
Open AccessPlausible nonsense and deliberative reasoning: Benchmarking LLMs against human judgment
Large Language Models (LLMs) are entering democratic contexts as instruments of governance, where the challenges at hand are ill-structured, marked by ambiguity and contestation. Ill-structured democratic problems demand more than factual precision; they call for intersubjective reasoning: context-sensitive judgments that others can understand and publicly accept. Using the Deliberative Reason Index (DRI), this study evaluates 60 LLMs against human deliberation across nine policy scenarios. Only four models consistently exceed the permutation-based null benchmark for alignment with human patterns of reason-giving. Most models fall short: their reason–preference structures rarely clear this threshold, even though their outputs can still appear coherent and persuasive. Yet outputs can appear reasonable even when this alignment is absent. The observed gap between surface plausibility and deliberative coherence urges caution: deploying LLMs in governance contexts requires prior assessment of their deliberative reasoning capacity, not just their surface outputs.
- Open Access
- 30 April 2026
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Human-Computer Interaction
- Vol. 10 (2) , 1-37
Bridging Instead of Replacing Online Coding Communities with AI through Community-Enriched Chatbot Designs CSCW008
LLM-based chatbots like ChatGPT have become popular tools for assisting with coding tasks. However, they often produce isolated responses and lack mechanisms for social learning or contextual grounding. In contrast, online coding communities like Kaggle offer socially mediated learning environments that foster critical thinking, engagement, and a sense of belonging. Yet, growing reliance on LLMs risks diminishing participation in these communities and weakening their collaborative value. To address this, we propose Community-Enriched AI, a design paradigm that embeds social learning dynamics into LLM-based chatbots by surfacing user-generated content and social design features from online coding communities. Using this paradigm, we implemented a RAG-based AI chatbot leveraging resources from Kaggle to validate our design. Across two empirical studies involving 28 and 12 data science learners, respectively, we found that Community-Enriched AI significantly enhances user trust, encourages engagement with community, and effectively supports learners in solving data science tasks. We conclude by discussing design implications for AI assistance systems that bridge—rather than replace—online coding communities.
- 30 April 2026
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Human-Computer Interaction
- Vol. 10 (2) , 1-37
Open AccessBridging Instead of Replacing Online Coding Communities with AI through Community-Enriched Chatbot Designs CSCW008
LLM-based chatbots like ChatGPT have become popular tools for assisting with coding tasks. However, they often produce isolated responses and lack mechanisms for social learning or contextual grounding. In contrast, online coding communities like Kaggle offer socially mediated learning environments that foster critical thinking, engagement, and a sense of belonging. Yet, growing reliance on LLMs risks diminishing participation in these communities and weakening their collaborative value. To address this, we propose Community-Enriched AI, a design paradigm that embeds social learning dynamics into LLM-based chatbots by surfacing user-generated content and social design features from online coding communities. Using this paradigm, we implemented a RAG-based AI chatbot leveraging resources from Kaggle to validate our design. Across two empirical studies involving 28 and 12 data science learners, respectively, we found that Community-Enriched AI significantly enhances user trust, encourages engagement with community, and effectively supports learners in solving data science tasks. We conclude by discussing design implications for AI assistance systems that bridge—rather than replace—online coding communities.
- 19 December 2025
- Published by The R Foundation
deliberr: Methods for Deliberation Analysis
An implementation of deliberative reasoning index (DRI) and related tools for analysis of deliberation survey data. Calculation of DRI, plot of intersubjective correlations (IC), generation of large-language model (LLM) survey data, and permutation tests are supported. Example datasets and a graphical user interface (GUI) are also available to support analysis. For more information, see Niemeyer and Veri (2022) .
- 19 December 2025
- Published by The R Foundation
deliberr: Methods for Deliberation Analysis
An implementation of deliberative reasoning index (DRI) and related tools for analysis of deliberation survey data. Calculation of DRI, plot of intersubjective correlations (IC), generation of large-language model (LLM) survey data, and permutation tests are supported. Example datasets and a graphical user interface (GUI) are also available to support analysis. For more information, see Niemeyer and Veri (2022) .
- Open Access
- 17 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 70-73
Technologies for Building Power in an Age of Democratic Backsliding: Advancing a Vision for People Power in CSCW Research
- 17 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 70-73
Open AccessTechnologies for Building Power in an Age of Democratic Backsliding: Advancing a Vision for People Power in CSCW Research
- Open Access
- 17 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 426-434
An Emergent Understanding of Human-AI Collaboration in Deliberation
- 17 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 426-434
Open AccessAn Emergent Understanding of Human-AI Collaboration in Deliberation
- Open Access
- 3 August 2025
- Published by Association for Computing Machinery (ACM)
- p. 83-100
Towards Designing Organizing Technologies for Increasing Participation and Representation in Open Democratic Innovations
- 3 August 2025
- Published by Association for Computing Machinery (ACM)
- p. 83-100
Open AccessTowards Designing Organizing Technologies for Increasing Participation and Representation in Open Democratic Innovations
- Open Access
- 13 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 152-158
datAR: A Situated Learning Approach for Data Literacy Through Everyday Objects
- 13 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 152-158
Open AccessdatAR: A Situated Learning Approach for Data Literacy Through Everyday Objects
- Open Access
- 2 May 2025
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Human-Computer Interaction
- Vol. 9 (2) , 1-29
DeliberationWorks: A Deliberation System for Developing Capacities in Civic Organizing
Civic technologies have helped activists mobilize large groups of people to complete simple actions like sharing a post on social media or signing an online petition. While mobilizing large numbers of people to complete low effort actions is important, mobilizing does not develop peoples' capacities to organize, which requires moving people up an engagement ladder to interdependently work with others on increasingly complex and challenging collective actions. Research on civic organizing suggests that deliberating with others about what collective actions to plan and complete is key to developing people's capacities to organize. In this paper, we explore whether deliberation can help organizers support potential activists in moving up the organizing engagement ladder. DeliberationWorks, a computer-supported deliberation system presents potential activists with background information on collective actions and intrapersonal deliberation questions, facilitates group discussion with experienced organizers, and prompts activists to fill out action plans for completing actions. Findings across two field deployments suggest that DeliberationWorks effectively helped organizers support potential activists in increasing their knowledge and interest in taking collective action, as well as successfully planning actions. Yet our findings also present a complex picture of additional learning challenges organizers encounter in deepening potential activists' engagement with organizing beyond the deliberation. We present four distinct engagement journeys based on participants' experiences during and after the deliberation to inform the design of future socio-technical interventions for moving potential activists further up the ladder. Our findings suggest that future systems designed to develop people's capacities to organize should help organizers invest in potential activists' capacities to increase engagement in the organization through 1-1 coaching and follow-up communications, based on understanding of their interests and needs from the deliberation. We contribute a novel approach that leverages organizing theory to design deliberation features to support organizers in increasing people's engagement with organizing, as well as evidence collected across two case study deployments that contribute a deepened understanding of new potential activists' needs in getting started with organizing.
- 2 May 2025
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Human-Computer Interaction
- Vol. 9 (2) , 1-29
Open AccessDeliberationWorks: A Deliberation System for Developing Capacities in Civic Organizing
Civic technologies have helped activists mobilize large groups of people to complete simple actions like sharing a post on social media or signing an online petition. While mobilizing large numbers of people to complete low effort actions is important, mobilizing does not develop peoples' capacities to organize, which requires moving people up an engagement ladder to interdependently work with others on increasingly complex and challenging collective actions. Research on civic organizing suggests that deliberating with others about what collective actions to plan and complete is key to developing people's capacities to organize. In this paper, we explore whether deliberation can help organizers support potential activists in moving up the organizing engagement ladder. DeliberationWorks, a computer-supported deliberation system presents potential activists with background information on collective actions and intrapersonal deliberation questions, facilitates group discussion with experienced organizers, and prompts activists to fill out action plans for completing actions. Findings across two field deployments suggest that DeliberationWorks effectively helped organizers support potential activists in increasing their knowledge and interest in taking collective action, as well as successfully planning actions. Yet our findings also present a complex picture of additional learning challenges organizers encounter in deepening potential activists' engagement with organizing beyond the deliberation. We present four distinct engagement journeys based on participants' experiences during and after the deliberation to inform the design of future socio-technical interventions for moving potential activists further up the ladder. Our findings suggest that future systems designed to develop people's capacities to organize should help organizers invest in potential activists' capacities to increase engagement in the organization through 1-1 coaching and follow-up communications, based on understanding of their interests and needs from the deliberation. We contribute a novel approach that leverages organizing theory to design deliberation features to support organizers in increasing people's engagement with organizing, as well as evidence collected across two case study deployments that contribute a deepened understanding of new potential activists' needs in getting started with organizing.
- Open Access
- 26 September 2023
- Published by Association for Computing Machinery (ACM)
- p. 197-205
Increasing Inclusion and Time-Efficiency in Participatory Policy-Making Deliberations with E-Scribing Technology
Citizens can increase openness, transparency, and accountability of institutions by taking part in face-to-face participatory policy-making deliberations, such as participatory budgeting assemblies. But for participants’ contributions to influence policy outcomes, organizers need to capture and synthesize participants’ input. Existing approaches are not inclusive for participants or require too much time from organizers. We designed e-scribing, a novel approach for capturing and synthesizing participants’ input from face-to-face deliberations in real time by combining scribes with digital technology. To evaluate the approach, we built DeliberationWorks, a digital deliberation technology that helps scribes (a) capture proxy input (i.e., as participants) that is complete and accurate so that participants do not need to interact with technology themselves and (b) synthesize the discussion in real time using labels. We deployed DeliberationWorks with 5 scribes in two face-to-face deliberations with 8-10 participants and found that, on average, 82% of the input was captured mostly accurately. After one hour of training, scribes synthesized input within 10 minutes of the end of the deliberation. Our findings suggest that e-scribing makes participatory policy-making more inclusive by allowing participants to share their input without interacting with technology, and more time-efficient by reducing synthesis and training times for organizers.
- 26 September 2023
- Published by Association for Computing Machinery (ACM)
- p. 197-205
Open AccessIncreasing Inclusion and Time-Efficiency in Participatory Policy-Making Deliberations with E-Scribing Technology
Citizens can increase openness, transparency, and accountability of institutions by taking part in face-to-face participatory policy-making deliberations, such as participatory budgeting assemblies. But for participants’ contributions to influence policy outcomes, organizers need to capture and synthesize participants’ input. Existing approaches are not inclusive for participants or require too much time from organizers. We designed e-scribing, a novel approach for capturing and synthesizing participants’ input from face-to-face deliberations in real time by combining scribes with digital technology. To evaluate the approach, we built DeliberationWorks, a digital deliberation technology that helps scribes (a) capture proxy input (i.e., as participants) that is complete and accurate so that participants do not need to interact with technology themselves and (b) synthesize the discussion in real time using labels. We deployed DeliberationWorks with 5 scribes in two face-to-face deliberations with 8-10 participants and found that, on average, 82% of the input was captured mostly accurately. After one hour of training, scribes synthesized input within 10 minutes of the end of the deliberation. Our findings suggest that e-scribing makes participatory policy-making more inclusive by allowing participants to share their input without interacting with technology, and more time-efficient by reducing synthesis and training times for organizers.