Luc Moreau

Provenance is a standardised record that describes how entities, activities, and agents have influenced a piece of data; it is commonly represented as graphs with relevant labels on both their nodes and edges. With the growing adoption of provenance in a wide range of application domains, users are increasingly confronted with an abundance of graph data, which may prove challenging to process. Graph kernels, on the other hand, have been successfully used to efficiently analyse graphs. In this paper, we introduce a novel graph kernel called provenance kernel, which is inspired by and tailored for provenance data. We employ provenance kernels to classify provenance graphs from three application domains. Our evaluation shows that they perform well in terms of classification accuracy and yield competitive results when compared against existing graph kernel methods and the provenance network analytics method while more efficient in computing time. Moreover, the provenance types used by provenance kernels are a symbolic representation of a tree pattern which can, in turn, be described using the domain-agnostic vocabulary of provenance. Therefore, provenance types thus allow for the creation of explanations of predictive models built on them.

As automated decision-making permeates almost all aspects of everyday life, capabilities to generate meaningful explanations for various stakeholders (i.e., decision-makers, addressees of decisions including individuals, auditors, and regulators) should be carefully deployed. This article presents a typology of explanations intended to support the first pillar of an explainability-by-design strategy. Its production has been achieved by pursuing a responsible innovation approach and introducing a new persona within the research and innovation process, i.e., a legal engineer, whose role is to work at the interface of two teams, the compliance and the engineering teams, and to oversee the process of requirement elicitation, which is often opinionated and narrowing. Once explanation requirements have been derived from applicable regulatory requirements, compliance rules, or business policies, they have been mapped to the dimensions of the typology to produce fine-grained explanation requirements, forming computable building blocks that can then be translated into system requirements during the technical design phase. The typology has been co-created with industry partners operating in two sectors: finance and education. Two pilot studies have thus been conducted to test both the feasibility of the generation and computation of explanations on the basis of the typology and the usefulness of the outputs in the light of the state-of-the-art. The typology comprises nine hierarchical dimensions. It can be leveraged to operate a stand-alone classifier of explanations that acts as detective controls within a broader partially automated compliance strategy. A machine-readable format of the typology is provided in the form of a light ontology.

As automated decision-making permeates almost all aspects of everyday life, capabilities to generate meaningful explanations for various stakeholders (i.e., decision-makers, addressees of decisions including individuals, auditors, and regulators) should be carefully deployed. This paper presents a typology of explanations intended to support the first pillar of an explainability-by-design strategy. Its production has been achieved by pursuing a responsible innovation approach and introducing a new persona within the research and innovation process, i.e., a legal engineer, whose role is to work at the interface of two teams, the compliance and the engineering teams and to oversee the process of requirement elicitation, which is often opinionated and narrowing. Once explanation requirements have been derived from applicable regulatory requirements, compliance rules or business policies, they have been mapped to the dimensions of the typology to produce fine-grained explanation requirements, forming computable building blocks that can then be translated into system requirements during the technical design phase. The typology has been co-created with industry partners operating in two sectors: finance and education. Two pilot studies have thus been conducted to test both the feasibility of the generation and computation of explanations on the basis of the typology and the usefulness of the outputs in the light of the state of the art. The typology comprises nine hierarchical dimensions. It can be leveraged to operate a stand-alone classifier of explanations that acts as detective controls within a broader partially-automated compliance strategy. A machine-readable format of the typology is provided in the form of a light ontology.

This research focuses on establishing trust in multiagent systems where human and AI agents collaborate. We propose a computational notion of actual trust, emphasising the modelling of an agent’s capacity to deliver tasks. Unlike reputation-based trust or performing a statistical analysis on past behaviour, our approach considers the specific setting in which agents interact. We integrate non-deterministic semantics for capturing inherent uncertainties within the behaviour of a multiagent system, but stress the importance of verifying an agent’s actual capabilities. We provide a conceptual analysis of actual trust’s characteristics and highlight relevant trust verification tools. By advancing the understanding and verification of trust in collaborative systems, this research contributes to responsible and trustworthy human-AI interactions, enhancing reliability in various domains.

People are increasingly interacting with machines embedded with intelligent decision aids, sometimes in high-stakes environments. When a human user comes into contact with a decision-making agent for the first time, it is likely that the agent’s behaviour or decisions do not precisely align with the human user’s goals. This phenomenon, known as goal alignment, has been recognised as a critical concern for human-machine teams. Prior work has focused on the effect of automation’s behavioural properties, such as predictability and reliability, on trust in human-machine interaction scenarios. However, little is known about situations where automation’s capabilities are misaligned with humans’ expectations and its impact on trust. Even less is known about the effect of environmental factors on trust. We study the relationship between intervention behaviours and trust in a simulated navigation task where the human user collaborates with an agent with misaligned goals. We evaluate trust quantitatively using intervention frequency as a behavioural measure and qualitatively using self-reports. By advancing the understanding and measurement of trust in collaborative settings, this research contributes to the development of trustworthy and symbiotic human-AI systems.

As autonomous systems (AS) increasingly become part of our daily lives, ensuring their trustworthiness is crucial. In order to demonstrate the trustworthiness of an AS, we first need to specify what is required for an AS to be considered trustworthy. This roadmap paper identifies key challenges for specifying for trustworthiness in AS, as identified during the "Specifying for Trustworthiness" workshop held as part of the UK Research and Innovation (UKRI) Trustworthy Autonomous Systems (TAS) programme. We look across a range of AS domains with consideration of the resilience, trust, functionality, verifiability, security, and governance and regulation of AS and identify some of the key specification challenges in these domains. We then highlight the intellectual challenges that are involved with specifying for trustworthiness in AS that cut across domains and are exacerbated by the inherent uncertainty involved with the environments in which AS need to operate.

Organisations have to comply with environmental regulations to protect the environment and meet internationally agreed climate change targets. To assist organisations, processes and standards are being defined to manage these compliance obligations. They typically rely on a notion of Environmental Management System (EMS), defined as a reflective framework allowing organisations to set and manage their goals, and demonstrate they follow due processes in order to comply with prevailing regulations. The importance of these obligations can be highlighted by the fact that failing to comply may lead to significant liabilities for organisations. An EMS framework, typically structured as a set of documents and spreadsheets, contains a record of continuously evolving regulations, teams, stakeholders, actions and updates. However, the maintainance of an EMS is often human driven, and therefore is error prone despite the meticulousness of environmental officers, and further requires external human auditing to check their validity. To avoid green washing, but also to contain the burden and cost of compliance, it is desirable for these claims to be checked by trusted automated means. Provenance is ideally suited to track the changes occurring in an EMS, allowing queries to determine precisely which compliance objective is prevailing at any point in time, whether it is being met, and who is responsible for it. Thus, this paper has a dual aim: first, it investigates the benefits of provenance for EMS, second, it presents the application of an emerging approach “Provenance-By-Design”, which automatically converts a specification of an EMS data model and its provenance to a data backend, a service for processing and querying of EMS provenance data, a client-side library to interact with such a service, and a simple user interface allowing developers to navigate the provenance. The application of a Provenance-By-Design approach to EMS applications results in novel opportunities for a provenance-based EMS; we present our preliminary reflection on their potential.