Luc Rocher
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Luc Rocher has a total of 49 co-authors. Here are the top co-authors by number of shared publications:
- Yves-Alexandre de Montjoye
Imperial College London
6 shared publications - Franziska Sofia Hafner
Oxford Internet Institute, University of Oxford, Oxford, UK
3 shared publications - Adam Mahdi
Oxford Internet Institute University of Oxford Oxford UK
2 shared publications - Andrew M. Bean
Oxford Internet Institute, University of Oxford, Oxford, UK
2 shared publications - Aruna S. Ekanayaka
Birmingham Women’s and Children’s NHS Foundation Trust, Birmingham, UK
2 shared publications Florimond Houssiau
Imperial College London, Exhibition Road, London SW7 2AZ, UK
2 shared publications
Latest Publications
- Open Access
- 15 October 2026
- Published by Elsevier in SSRN Electronic Journal
LLM Adoption is Eroding Evidence in Online Research
Simulating human participants using large language models (LLMs) has gained traction among researchers. In parallel, participants are also turning to LLMs to complete surveys faster, raising alarms about the degrading quality of crowdsourced data. Both researchers and crowd-workers are motivated by the same objectives of convenience and efficiency. Yet, LLM use by researchers is typically incentivized and praised as innovative, whereas LLM use by crowd-workers is surveilled and condemned as fraud. We argue that this double standard rests on a failure to recognize the inherent limitations of LLM simulations as well as the ongoing displacement and surveillance of crowd-workers. Overall, racing to detect LLM use among participants while simultaneously using LLMs to simulate them risks eroding the authenticity of data available to study human opinions and behaviors.
- 15 October 2026
- Published by Elsevier in SSRN Electronic Journal
Open AccessLLM Adoption is Eroding Evidence in Online Research
Simulating human participants using large language models (LLMs) has gained traction among researchers. In parallel, participants are also turning to LLMs to complete surveys faster, raising alarms about the degrading quality of crowdsourced data. Both researchers and crowd-workers are motivated by the same objectives of convenience and efficiency. Yet, LLM use by researchers is typically incentivized and praised as innovative, whereas LLM use by crowd-workers is surveilled and condemned as fraud. We argue that this double standard rests on a failure to recognize the inherent limitations of LLM simulations as well as the ongoing displacement and surveillance of crowd-workers. Overall, racing to detect LLM use among participants while simultaneously using LLMs to simulate them risks eroding the authenticity of data available to study human opinions and behaviors.
- Top Cited PapersOpen Access
- 29 April 2026
- Published by Springer Nature in Nature
- Vol. 652 (8112) , 1159-1165
Training language models to be warm can reduce accuracy and increase sycophancy
Artificial intelligence developers are increasingly building language models with warm and friendly personas that millions of people now use for advice, therapy and companionship 1 . Here we show how this can create a significant trade-off: optimizing language models for warmth can undermine their performance, especially when users express vulnerability. We conducted controlled experiments on five different language models, training them to produce warmer responses, then evaluating them on consequential tasks. Warm models showed substantially higher error rates (+10 to +30 percentage points) than their original counterparts, promoting conspiracy theories, providing inaccurate factual information and offering incorrect medical advice. They were also significantly more likely to validate incorrect user beliefs, particularly when user messages expressed feelings of sadness. Importantly, these effects were consistent across different model architectures, and occurred despite preserved performance on standard tests, revealing systematic risks that standard testing practices may fail to detect. Our findings suggest that training artificial intelligence systems to be warm may come at a cost to accuracy, and that warmth and accuracy may not be independent by default. As these systems are deployed at an unprecedented scale and take on intimate roles in people’s lives, this trade-off warrants attention from developers, policymakers and users alike.
- 29 April 2026
- Published by Springer Nature in Nature
- Vol. 652 (8112) , 1159-1165
Top Cited PapersOpen AccessTraining language models to be warm can reduce accuracy and increase sycophancy
Artificial intelligence developers are increasingly building language models with warm and friendly personas that millions of people now use for advice, therapy and companionship 1 . Here we show how this can create a significant trade-off: optimizing language models for warmth can undermine their performance, especially when users express vulnerability. We conducted controlled experiments on five different language models, training them to produce warmer responses, then evaluating them on consequential tasks. Warm models showed substantially higher error rates (+10 to +30 percentage points) than their original counterparts, promoting conspiracy theories, providing inaccurate factual information and offering incorrect medical advice. They were also significantly more likely to validate incorrect user beliefs, particularly when user messages expressed feelings of sadness. Importantly, these effects were consistent across different model architectures, and occurred despite preserved performance on standard tests, revealing systematic risks that standard testing practices may fail to detect. Our findings suggest that training artificial intelligence systems to be warm may come at a cost to accuracy, and that warmth and accuracy may not be independent by default. As these systems are deployed at an unprecedented scale and take on intimate roles in people’s lives, this trade-off warrants attention from developers, policymakers and users alike.
- Top Cited PapersOpen Access
- 17 April 2026
- Published by Springer Nature in Nature Medicine
- Vol. 32 (5) , 1937
Publisher Correction: Reliability of LLMs as medical assistants for the general public: a randomized preregistered study
- 17 April 2026
- Published by Springer Nature in Nature Medicine
- Vol. 32 (5) , 1937
Top Cited PapersOpen AccessPublisher Correction: Reliability of LLMs as medical assistants for the general public: a randomized preregistered study
- Top Cited PapersOpen Access
- 9 February 2026
- Published by Springer Nature in Nature Medicine
- Vol. 32 (2) , 609-615
Reliability of LLMs as medical assistants for the general public: a randomized preregistered study
Global healthcare providers are exploring the use of large language models (LLMs) to provide medical advice to the public. LLMs now achieve nearly perfect scores on medical licensing exams, but this does not necessarily translate to accurate performance in real-world settings. We tested whether LLMs can assist members of the public in identifying underlying conditions and choosing a course of action (disposition) in ten medical scenarios in a controlled study with 1,298 participants. Participants were randomly assigned to receive assistance from an LLM (GPT-4o, Llama 3, Command R+) or a source of their choice (control). Tested alone, LLMs complete the scenarios accurately, correctly identifying conditions in 94.9% of cases and disposition in 56.3% on average. However, participants using the same LLMs identified relevant conditions in fewer than 34.5% of cases and disposition in fewer than 44.2%, both no better than the control group. We identify user interactions as a challenge to the deployment of LLMs for medical advice. Standard benchmarks for medical knowledge and simulated patient interactions do not predict the failures we find with human participants. Moving forward, we recommend systematic human user testing to evaluate interactive capabilities before public deployments in healthcare.
- 9 February 2026
- Published by Springer Nature in Nature Medicine
- Vol. 32 (2) , 609-615
Top Cited PapersOpen AccessReliability of LLMs as medical assistants for the general public: a randomized preregistered study
Global healthcare providers are exploring the use of large language models (LLMs) to provide medical advice to the public. LLMs now achieve nearly perfect scores on medical licensing exams, but this does not necessarily translate to accurate performance in real-world settings. We tested whether LLMs can assist members of the public in identifying underlying conditions and choosing a course of action (disposition) in ten medical scenarios in a controlled study with 1,298 participants. Participants were randomly assigned to receive assistance from an LLM (GPT-4o, Llama 3, Command R+) or a source of their choice (control). Tested alone, LLMs complete the scenarios accurately, correctly identifying conditions in 94.9% of cases and disposition in 56.3% on average. However, participants using the same LLMs identified relevant conditions in fewer than 34.5% of cases and disposition in fewer than 44.2%, both no better than the control group. We identify user interactions as a challenge to the deployment of LLMs for medical advice. Standard benchmarks for medical knowledge and simulated patient interactions do not predict the failures we find with human participants. Moving forward, we recommend systematic human user testing to evaluate interactive capabilities before public deployments in healthcare.
- 6 February 2026
- Published by Springer Nature in Nature Machine Intelligence
- Vol. 8 (3) , 295
Attributing and situating knowledge cannot be left to language models
- 6 February 2026
- Published by Springer Nature in Nature Machine Intelligence
- Vol. 8 (3) , 295
Attributing and situating knowledge cannot be left to language models
- Open Access
- 14 November 2025
- Published by Springer Nature in AI & SOCIETY
- Vol. 41 (4) , 3413-3435
‘We can see a savage’: a case study of the colonial gaze in generative AI algorithms
Theorizing the failures of computer vision algorithms requires shifting from detecting and fixing biases towards understanding how algorithms are shaped by social, historical, and political real-world precursors. To better understand the socially embedded and historically rooted representational harms of these algorithms, we analyze how AI image captioning depicts archival images of living ethnological exibitions (so-called 'human zoos'), mass stereotype-producing public exhibitions of colonized people common in Europe and the US from the 1870s to the 1930s, which were meant to symbolize the imagined superiority of Western societies and justify their colonial violence. We collected and analyzed more than 3800 captions from 100 archival images using MidJourney––a modern, state-of-the-art generative AI platform. Combining quantification with close reading of the captions, we found evidence of a ‘colonial gaze,’ an epistemological viewpoint from the perspective of colonizers characterized by significant representational harms representing five main themes: essentialism (41.6% of captions), cultural erasure (54.5%), dehumanization (11.1%), othering (28.4%), and infantilization (26.8%), with striking parallels between AI-generated captions and the original framings of human zoos informed by a broader colonial epistemology. Based on this analysis, we propose to conceptualize the colonial gaze in generative AI as an automated process of object identification and relational interpretation that draws on historical visual tropes and hierarchical logics rooted in colonial epistemologies. Trigger warning: This article contains extremely racialized text and images produced by both colonizers and the machines.
- 14 November 2025
- Published by Springer Nature in AI & SOCIETY
- Vol. 41 (4) , 3413-3435
Open Access‘We can see a savage’: a case study of the colonial gaze in generative AI algorithms
Theorizing the failures of computer vision algorithms requires shifting from detecting and fixing biases towards understanding how algorithms are shaped by social, historical, and political real-world precursors. To better understand the socially embedded and historically rooted representational harms of these algorithms, we analyze how AI image captioning depicts archival images of living ethnological exibitions (so-called 'human zoos'), mass stereotype-producing public exhibitions of colonized people common in Europe and the US from the 1870s to the 1930s, which were meant to symbolize the imagined superiority of Western societies and justify their colonial violence. We collected and analyzed more than 3800 captions from 100 archival images using MidJourney––a modern, state-of-the-art generative AI platform. Combining quantification with close reading of the captions, we found evidence of a ‘colonial gaze,’ an epistemological viewpoint from the perspective of colonizers characterized by significant representational harms representing five main themes: essentialism (41.6% of captions), cultural erasure (54.5%), dehumanization (11.1%), othering (28.4%), and infantilization (26.8%), with striking parallels between AI-generated captions and the original framings of human zoos informed by a broader colonial epistemology. Based on this analysis, we propose to conceptualize the colonial gaze in generative AI as an automated process of object identification and relational interpretation that draws on historical visual tropes and hierarchical logics rooted in colonial epistemologies. Trigger warning: This article contains extremely racialized text and images produced by both colonizers and the machines.
- Open Access
- 16 October 2025
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Human-Computer Interaction
- Vol. 9 (7) , 1-31
Into the Crossfire: Evaluating the Use of a Language Model to Crowdsource Gun Violence Reports
Gun violence is a pressing human rights issue that affects nearly every dimension of the social fabric, from healthcare and education to psychology and the economy. Reliable data on firearm events is paramount to developing more effective public policy and emergency responses. However, the lack of comprehensive databases and the risks of in-person surveys prevent human rights organizations from collecting needed data in most countries. Here, we partner with a Brazilian human rights organization to conduct a systematic evaluation of language models to assist with monitoring real-world firearm events from social media data. We propose a fine-tuned BERT-based model trained on Twitter (now X) texts to distinguish gun violence reports from ordinary Portuguese texts. We then incorporate our model into a web application and test it in a live intervention. We study and interview Brazilian analysts who continuously check social media texts to identify new gun violence events. Qualitative assessments show that our solution helped all analysts use their time more efficiently and expanded their search capacities. Quantitative assessments show that the use of our model was associated with analysts having further interactions with online users reporting gun violence. Our findings suggest that human-centered interventions using language models can help support the work of human rights organizations.
- 16 October 2025
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Human-Computer Interaction
- Vol. 9 (7) , 1-31
Open AccessInto the Crossfire: Evaluating the Use of a Language Model to Crowdsource Gun Violence Reports
Gun violence is a pressing human rights issue that affects nearly every dimension of the social fabric, from healthcare and education to psychology and the economy. Reliable data on firearm events is paramount to developing more effective public policy and emergency responses. However, the lack of comprehensive databases and the risks of in-person surveys prevent human rights organizations from collecting needed data in most countries. Here, we partner with a Brazilian human rights organization to conduct a systematic evaluation of language models to assist with monitoring real-world firearm events from social media data. We propose a fine-tuned BERT-based model trained on Twitter (now X) texts to distinguish gun violence reports from ordinary Portuguese texts. We then incorporate our model into a web application and test it in a live intervention. We study and interview Brazilian analysts who continuously check social media texts to identify new gun violence events. Qualitative assessments show that our solution helped all analysts use their time more efficiently and expanded their search capacities. Quantitative assessments show that the use of our model was associated with analysts having further interactions with online users reporting gun violence. Our findings suggest that human-centered interventions using language models can help support the work of human rights organizations.
- Open Access
Training language models to be warm and empathetic makes them less reliable and more sycophantic
Artificial intelligence (AI) developers are increasingly building language models with warm and empathetic personas that millions of people now use for advice, therapy, and companionship. Here, we show how this creates a significant trade-off: optimizing language models for warmth undermines their reliability, especially when users express vulnerability. We conducted controlled experiments on five language models of varying sizes and architectures, training them to produce warmer, more empathetic responses, then evaluating them on safety-critical tasks. Warm models showed substantially higher error rates (+10 to +30 percentage points) than their original counterparts, promoting conspiracy theories, providing incorrect factual information, and offering problematic medical advice. They were also significantly more likely to validate incorrect user beliefs, particularly when user messages expressed sadness. Importantly, these effects were consistent across different model architectures, and occurred despite preserved performance on standard benchmarks, revealing systematic risks that current evaluation practices may fail to detect. As human-like AI systems are deployed at an unprecedented scale, our findings indicate a need to rethink how we develop and oversee these systems that are reshaping human relationships and social interaction.
Open AccessTraining language models to be warm and empathetic makes them less reliable and more sycophantic
Artificial intelligence (AI) developers are increasingly building language models with warm and empathetic personas that millions of people now use for advice, therapy, and companionship. Here, we show how this creates a significant trade-off: optimizing language models for warmth undermines their reliability, especially when users express vulnerability. We conducted controlled experiments on five language models of varying sizes and architectures, training them to produce warmer, more empathetic responses, then evaluating them on safety-critical tasks. Warm models showed substantially higher error rates (+10 to +30 percentage points) than their original counterparts, promoting conspiracy theories, providing incorrect factual information, and offering problematic medical advice. They were also significantly more likely to validate incorrect user beliefs, particularly when user messages expressed sadness. Importantly, these effects were consistent across different model architectures, and occurred despite preserved performance on standard benchmarks, revealing systematic risks that current evaluation practices may fail to detect. As human-like AI systems are deployed at an unprecedented scale, our findings indicate a need to rethink how we develop and oversee these systems that are reshaping human relationships and social interaction.
- Open Access
- 23 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 1677-1695
Gender Trouble in Language Models: An Empirical Audit Guided by Gender Performativity Theory
- 23 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 1677-1695
Open AccessGender Trouble in Language Models: An Empirical Audit Guided by Gender Performativity Theory
- Open Access
- 25 April 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-31
Access Denied: Meaningful Data Access for Quantitative Algorithm Audits
- 25 April 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-31
Open AccessAccess Denied: Meaningful Data Access for Quantitative Algorithm Audits