Societal and technological progress as sewing an ever-growing, ever-changing, patchy, and polychrome quilt
Authors:
Preprint
Abstract
Artificial Intelligence (AI) systems are increasingly placed in positions where their decisions have real consequences, e.g., moderating online spaces, conducting research, and advising on policy. Ensuring they operate in a safe and ethically acceptable fashion is thus critical. However, most solutions have been a form of one-size-fits-all "alignment". We are worried that such systems, which overlook enduring moral diversity, will spark resistance, erode trust, and destabilize our institutions. This paper traces the underlying problem to an often-unstated Axiom of Rational Convergence: the idea that under ideal conditions, rational agents will converge in the limit of conversation on a single ethics. Treating that premise as both optional and doubtful, we propose what we call the appropriateness framework: an alternative approach grounded in conflict theory, cultural evolution, multi-agent systems, and institutional economics. The appropriateness framework treats persistent disagreement as the normal case and designs for it by applying four principles: (1) contextual grounding, (2) community customization, (3) continual adaptation, and (4) polycentric governance. We argue here that adopting these design principles is a good way to shift the main alignment metaphor from moral unification to a more productive metaphor of conflict management, and that taking this step is both desirable and urgent.Keywords
All Related Versions
This publication has 26 references indexed in Scilit:
- Tradition and invention: The bifocal stance theory of cultural evolutionBehavioral and Brain Sciences, 2022
- Cultural evolution: Is causal inference the secret of our success?Current Biology, 2021
- The role of causal knowledge in the evolution of traditional technologyCurrent Biology, 2021
- Causal understanding is not necessary for the improvement of culturally evolving technologyNature Human Behaviour, 2019
- The Control Problem. Excerpts fromSuperintelligence: Paths, Dangers, Strategies2016
- The cultural niche: Why social learning is essential for human adaptationProceedings of the National Academy of Sciences, 2011
- Polycentric systems for coping with collective action and global environmental changeGlobal Environmental Change, 2010
- Liberals and conservatives rely on different sets of moral foundations.Journal of Personality and Social Psychology, 2009
- Sacred bounds on rational resolution of violent political conflictProceedings of the National Academy of Sciences, 2007
- “Economic man” in cross-cultural perspective: Behavioral experiments in 15 small-scale societiesBehavioral and Brain Sciences, 2005