Marina De Vos

Background: Value alignment in computer science research is often used to refer to the process of aligning the behaviour of artificial intelligence systems with humans’ desires, but the way the phrase is used often lacks precision. Objectives: In this paper, we conduct a systematic literature review to advance the understanding of value alignment in artificial intelligence by characterising the topic in the context of its research literature. We use this to suggest a more precise definition of the term. Methods: We analyse the abstracts, introductions and conclusions of 172 value alignment research articles that have been published in recent years and synthesise their content using thematic analysis. From these 172 papers we select 85 papers using a structured criteria for a deep analysis, coding these papers in full. Results: Our analysis leads to six themes: value alignment drivers & approaches; challenges in value alignment; values in value alignment; cognitive processes in humans and AI; human-agent teaming; and designing and developing value-aligned systems. Conclusions: By analysing these themes in the context of the literature, we define value alignment as an ongoing process between humans and autonomous agents that aims to express and implement abstract values in diverse contexts, while managing the cognitive limits of both humans and AI agents and also balancing the conflicting ethical and political demands generated by the values in different groups. Our analysis gives rise to a set of research challenges and opportunities in the field of value alignment for future work.

In a normative environment, an agent’s actions are directed not only by its goals but also by the norms activated by its actions and those of other actors. The potential for conflict between agent goals and norms makes decision making challenging, in that it requires looking ahead to consider the longer-term consequences of which goal to satisfy or which norm to comply with in face of conflict. We therefore seek to determine the actions an agent should select at each point in time, taking account of its temporal goals, norms, and their conflicts. We propose a solution in which a normative planning problem is the basis for practical reasoning based on argumentation. Various types of conflict within goals, within norms, and between goals and norms are identified based on temporal properties of these entities. The properties of the best plan(s) with respect to goal achievement and norm compliance are mapped to arguments, followed by mapping their conflicts to attack between arguments, all of which are used to identify why a plan is justified.

Norms are utilised in agent societies to encourage acceptable behaviour by the participating agents. They can be established or revised from the top-down (authority) or from the bottom-up (populace). The study of norm creation from the bottom-up—or norm emergence/convergence—shows evidence of increasing activity. In consequence, we seek to analyse and categorize the approaches proposed in the literature for facilitating norm emergence. This paper makes three contributions to the study of norm emergence. Firstly, we present the different perspectives of norms and their impact on the norm emergence process, with the aim of comparing their similarities and differences in implementing the norm life cycle. Secondly, we identify the characteristics that support norm emergence that are observed in the emergence literature. Finally, we identify and propose future topics for study for the community, through a discussion of the challenges and opportunities in norm emergence.

Normative capabilities in multi-agent systems (MAS) can be represented within agents, separately as institutions, or a blend of the two. This paper addresses how to extend the principles of open MAS to the provision of normative reasoning capabilities, which are currently either embedded in existing MAS platforms - tightly coupled and inaccessible - or not present. We use a resource-oriented architecture (ROA) pattern, that we call deontic sensors, to make normative reasoning part of an open MAS architecture. The pattern specifies how to loosely couple MAS and normative frameworks, such that each is agnostic of the other, while augmenting the brute facts that an agent perceives with institutional facts, that capture each institution's interpretation of an agent's action. In consequence, a MAS without normative capabilities can acquire them, and an embedded normative framework can be de-coupled and opened to other MAS platforms. More importantly, the deontic sensor pattern allows normative reasoning to be published as services, opening routes to certification and re-use, creation of (formalized) trust and non-specialist access to "on demand'' normative reasoning.

Prediction markets are well-established tools for aggregating information from diverse sources into accurate forecasts. Their success has been demonstrated in a wide range applications, including presidential campaigns, sporting events, and economic outcomes. Recently, they've been introduced to the machine learning community in the form of artificial prediction markets, in which algorithms trade contracts reflecting their levels of confidence. To date, these markets have mostly been studied in the context of offline classification, with promising results. The authors extend them to enable their use in online regression and introduce adaptive trading strategies informed by individual trading history and the ability of participants to revise their predictions by reflecting on the wisdom of the crowd, which is manifested in the collective performance of the market. The authors empirically evaluate their model using multiple datasets and show that it outperforms several well-established techniques from the literature on online regression.

Autonomous software agents operating in dynamic environments need to constantly reason about actions in pursuit of their goals, while taking into consideration norms which might be imposed on those actions. Normative practical reasoning supports agents making decisions about what is best for them to (not) do in a given situation. What makes practical reasoning challenging is the interplay between goals that agents are pursuing and the norms that the agents are trying to uphold. We offer a formalisation to allow agents to plan for multiple goals and norms in the presence of durative actions that can be executed concurrently. We compare plans based on decision-theoretic notions (i.e. utility) such that the utility gain of goals and utility loss of norm violations are the basis for this comparison. The set of optimal plans consists of plans that maximise the overall utility, each of which can be chosen by the agent to execute. We provide an implementation of our proposal in Answer Set Programming, thus allowing us to state the original problem in terms of a logic program that can be queried for solutions with specific properties. The implementation is proven to be sound and complete.