Carina Prunkl
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
Carina Prunkl has a total of 25 co-authors. Here are the top co-authors by number of shared publications:
Allan Dafoe
Future of Humanity Institute University of Oxford
2 shared publications- Andres Algaba
Vrije Universiteit Brussel
2 shared publications - Carmen Mazijn
Vrije Universiteit Brussel
2 shared publications Carolyn Ashurst
Future of Humanity Institute University of Oxford
2 shared publications- Jan Danckaert
Vrije Universiteit Brussel
2 shared publications Jan Leike
Future of Humanity Institute University of Oxford
2 shared publications
Latest Publications
- Open Access
- 21 June 2026
- Published by Association for Computing Machinery (ACM)
- p. 573-587
Agency in Child–AI Interaction: A Review of How It Is Conceptualised, Studied, and Supported in HCI
- 21 June 2026
- Published by Association for Computing Machinery (ACM)
- p. 573-587
Open AccessAgency in Child–AI Interaction: A Review of How It Is Conceptualised, Studied, and Supported in HCI
- 25 November 2025
- Published by Springer Nature in Ethics and Information Technology
- Vol. 28 (1) , 7
Examining popular arguments against AI existential risk: a philosophical analysis
- 25 November 2025
- Published by Springer Nature in Ethics and Information Technology
- Vol. 28 (1) , 7
Examining popular arguments against AI existential risk: a philosophical analysis
- 9 August 2024
- Published by Springer Nature in Nature Methods
- Vol. 21 (8) , 1407-1408
AI meets biology: a call for community governance
- 9 August 2024
- Published by Springer Nature in Nature Methods
- Vol. 21 (8) , 1407-1408
AI meets biology: a call for community governance
- Top Cited PapersOpen Access
- 24 June 2024
- Published by Springer Nature in Minds and Machines
- Vol. 34 (3) , 1-21
Human Autonomy at Risk? An Analysis of the Challenges from AI
Autonomy is a core value that is deeply entrenched in the moral, legal, and political practices of many societies. The development and deployment of artificial intelligence (AI) have raised new questions about AI’s impacts on human autonomy. However, systematic assessments of these impacts are still rare and often held on a case-by-case basis. In this article, I provide a conceptual framework that both ties together seemingly disjoint issues about human autonomy, as well as highlights differences between them. In the first part, I distinguish between distinct concerns that are currently addressed under the umbrella term ‘human autonomy’. In particular, I show how differentiating between autonomy-as-authenticity and autonomy-as-agency helps us to pinpoint separate challenges from AI deployment. Some of these challenges are already well-known (e.g. online manipulation or limitation of freedom), whereas others have received much less attention (e.g. adaptive preference formation). In the second part, I address the different roles AI systems can assume in the context of autonomy. In particular, I differentiate between AI systems taking on agential roles and AI systems being used as tools. I conclude that while there is no ‘silver bullet’ to address concerns about human autonomy, considering its various dimensions can help us to systematically address the associated risks.
- 24 June 2024
- Published by Springer Nature in Minds and Machines
- Vol. 34 (3) , 1-21
Top Cited PapersOpen AccessHuman Autonomy at Risk? An Analysis of the Challenges from AI
Autonomy is a core value that is deeply entrenched in the moral, legal, and political practices of many societies. The development and deployment of artificial intelligence (AI) have raised new questions about AI’s impacts on human autonomy. However, systematic assessments of these impacts are still rare and often held on a case-by-case basis. In this article, I provide a conceptual framework that both ties together seemingly disjoint issues about human autonomy, as well as highlights differences between them. In the first part, I distinguish between distinct concerns that are currently addressed under the umbrella term ‘human autonomy’. In particular, I show how differentiating between autonomy-as-authenticity and autonomy-as-agency helps us to pinpoint separate challenges from AI deployment. Some of these challenges are already well-known (e.g. online manipulation or limitation of freedom), whereas others have received much less attention (e.g. adaptive preference formation). In the second part, I address the different roles AI systems can assume in the context of autonomy. In particular, I differentiate between AI systems taking on agential roles and AI systems being used as tools. I conclude that while there is no ‘silver bullet’ to address concerns about human autonomy, considering its various dimensions can help us to systematically address the associated risks.
- Top Cited PapersOpen Access
- 5 February 2024
- Published by Springer Nature in Philosophical Studies
- Vol. 182 (1) , 185-203
Algorithmic profiling as a source of hermeneutical injustice
It is well-established that algorithms can be instruments of injustice. It is less frequently discussed, however, how current modes of AI deployment often make the very discovery of injustice difficult, if not impossible. In this article, we focus on the effects of algorithmic profiling on epistemic agency. We show how algorithmic profiling can give rise to epistemic injustice through the depletion of epistemic resources that are needed to interpret and evaluate certain experiences. By doing so, we not only demonstrate how the philosophical conceptual framework of epistemic injustice can help pinpoint potential, systematic harms from algorithmic profiling, but we also identify a novel source of hermeneutical injustice that to date has received little attention in the relevant literature, what we call epistemic fragmentation. As we detail in this paper, epistemic fragmentation is a structural characteristic of algorithmically-mediated environments that isolate individuals, making it more difficult to develop, uptake and apply new epistemic resources, thus making it more difficult to identify and conceptualise emerging harms in these environments. We thus trace the occurrence of hermeneutical injustice back to the fragmentation of the epistemic experiences of individuals, who are left more vulnerable by the inability to share, compare and learn from shared experiences.
- 5 February 2024
- Published by Springer Nature in Philosophical Studies
- Vol. 182 (1) , 185-203
Top Cited PapersOpen AccessAlgorithmic profiling as a source of hermeneutical injustice
It is well-established that algorithms can be instruments of injustice. It is less frequently discussed, however, how current modes of AI deployment often make the very discovery of injustice difficult, if not impossible. In this article, we focus on the effects of algorithmic profiling on epistemic agency. We show how algorithmic profiling can give rise to epistemic injustice through the depletion of epistemic resources that are needed to interpret and evaluate certain experiences. By doing so, we not only demonstrate how the philosophical conceptual framework of epistemic injustice can help pinpoint potential, systematic harms from algorithmic profiling, but we also identify a novel source of hermeneutical injustice that to date has received little attention in the relevant literature, what we call epistemic fragmentation. As we detail in this paper, epistemic fragmentation is a structural characteristic of algorithmically-mediated environments that isolate individuals, making it more difficult to develop, uptake and apply new epistemic resources, thus making it more difficult to identify and conceptualise emerging harms in these environments. We thus trace the occurrence of hermeneutical injustice back to the fragmentation of the epistemic experiences of individuals, who are left more vulnerable by the inability to share, compare and learn from shared experiences.
- 21 October 2023
- Published by Springer Nature
- p. 346-367
LUCID–GAN: Conditional Generative Models to Locate Unfairness
- 21 October 2023
- Published by Springer Nature
- p. 346-367
LUCID–GAN: Conditional Generative Models to Locate Unfairness
- Open Access
- 26 June 2023
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 37 (12) , 14391-14399
LUCID: Exposing Algorithmic Bias through Inverse Design
AI systems can create, propagate, support, and automate bias in decision-making processes. To mitigate biased decisions, we both need to understand the origin of the bias and define what it means for an algorithm to make fair decisions. Most group fairness notions assess a model's equality of outcome by computing statistical metrics on the outputs. We argue that these output metrics encounter intrinsic obstacles and present a complementary approach that aligns with the increasing focus on equality of treatment. By Locating Unfairness through Canonical Inverse Design (LUCID), we generate a canonical set that shows the desired inputs for a model given a preferred output. The canonical set reveals the model's internal logic and exposes potential unethical biases by repeatedly interrogating the decision-making process. We evaluate LUCID on the UCI Adult and COMPAS data sets and find that some biases detected by a canonical set differ from those of output metrics. The results show that by shifting the focus towards equality of treatment and looking into the algorithm's internal workings, the canonical sets are a valuable addition to the toolbox of algorithmic fairness evaluation.
- 26 June 2023
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 37 (12) , 14391-14399
Open AccessLUCID: Exposing Algorithmic Bias through Inverse Design
AI systems can create, propagate, support, and automate bias in decision-making processes. To mitigate biased decisions, we both need to understand the origin of the bias and define what it means for an algorithm to make fair decisions. Most group fairness notions assess a model's equality of outcome by computing statistical metrics on the outputs. We argue that these output metrics encounter intrinsic obstacles and present a complementary approach that aligns with the increasing focus on equality of treatment. By Locating Unfairness through Canonical Inverse Design (LUCID), we generate a canonical set that shows the desired inputs for a model given a preferred output. The canonical set reveals the model's internal logic and exposes potential unethical biases by repeatedly interrogating the decision-making process. We evaluate LUCID on the UCI Adult and COMPAS data sets and find that some biases detected by a canonical set differ from those of output metrics. The results show that by shifting the focus towards equality of treatment and looking into the algorithm's internal workings, the canonical sets are a valuable addition to the toolbox of algorithmic fairness evaluation.
- 15 November 2022
- Published by Springer Nature
- p. 67-71
Is There a Trade-Off Between Human Autonomy and the ‘Autonomy’ of AI Systems?
- 15 November 2022
- Published by Springer Nature
- p. 67-71
Is There a Trade-Off Between Human Autonomy and the ‘Autonomy’ of AI Systems?
- 23 February 2022
- Published by Springer Nature in Nature Machine Intelligence
- Vol. 4 (2) , 99-101
Human autonomy in the age of artificial intelligence
- 23 February 2022
- Published by Springer Nature in Nature Machine Intelligence
- Vol. 4 (2) , 99-101
Human autonomy in the age of artificial intelligence
- Open Access
Institutionalising Ethics in AI through Broader Impact Requirements
Turning principles into practice is one of the most pressing challenges of artificial intelligence (AI) governance. In this article, we reflect on a novel governance initiative by one of the world's largest AI conferences. In 2020, the Conference on Neural Information Processing Systems (NeurIPS) introduced a requirement for submitting authors to include a statement on the broader societal impacts of their research. Drawing insights from similar governance initiatives, including institutional review boards (IRBs) and impact requirements for funding applications, we investigate the risks, challenges and potential benefits of such an initiative. Among the challenges, we list a lack of recognised best practice and procedural transparency, researcher opportunity costs, institutional and social pressures, cognitive biases, and the inherently difficult nature of the task. The potential benefits, on the other hand, include improved anticipation and identification of impacts, better communication with policy and governance experts, and a general strengthening of the norms around responsible research. To maximise the chance of success, we recommend measures to increase transparency, improve guidance, create incentives to engage earnestly with the process, and facilitate public deliberation on the requirement's merits and future. Perhaps the most important contribution from this analysis are the insights we can gain regarding effective community-based governance and the role and responsibility of the AI research community more broadly.
Open AccessInstitutionalising Ethics in AI through Broader Impact Requirements
Turning principles into practice is one of the most pressing challenges of artificial intelligence (AI) governance. In this article, we reflect on a novel governance initiative by one of the world's largest AI conferences. In 2020, the Conference on Neural Information Processing Systems (NeurIPS) introduced a requirement for submitting authors to include a statement on the broader societal impacts of their research. Drawing insights from similar governance initiatives, including institutional review boards (IRBs) and impact requirements for funding applications, we investigate the risks, challenges and potential benefits of such an initiative. Among the challenges, we list a lack of recognised best practice and procedural transparency, researcher opportunity costs, institutional and social pressures, cognitive biases, and the inherently difficult nature of the task. The potential benefits, on the other hand, include improved anticipation and identification of impacts, better communication with policy and governance experts, and a general strengthening of the norms around responsible research. To maximise the chance of success, we recommend measures to increase transparency, improve guidance, create incentives to engage earnestly with the process, and facilitate public deliberation on the requirement's merits and future. Perhaps the most important contribution from this analysis are the insights we can gain regarding effective community-based governance and the role and responsibility of the AI research community more broadly.