From Transparency to Accountability and Back: A Discussion of Access and Evidence in AI Auditing
Preprint
Abstract
Artificial intelligence (AI) is increasingly intervening in our lives, raising widespread concern about its unintended and undeclared side effects. These developments have brought attention to the problem of AI auditing: the systematic evaluation and analysis of an AI system, its development, and its behavior relative to a set of predetermined criteria. Auditing can take many forms, including pre-deployment risk assessments, ongoing monitoring, and compliance testing. It plays a critical role in providing assurances to various AI stakeholders, from developers to end users. Audits may, for instance, be used to verify that an algorithm complies with the law, is consistent with industry standards, and meets the developer's claimed specifications. However, there are many operational challenges to AI auditing that complicate its implementation. In this work, we examine a key operational issue in AI auditing: what type of access to an AI system is needed to perform a meaningful audit? Addressing this question has direct policy relevance, as it can inform AI audit guidelines and requirements. We begin by discussing the factors that auditors balance when determining the appropriate type of access, and unpack the benefits and drawbacks of four types of access. We conclude that, at minimum, black-box access -- providing query access to a model without exposing its internal implementation -- should be granted to auditors, as it balances concerns related to trade secrets, data privacy, audit standardization, and audit efficiency. We then suggest a framework for determining how much further access (in addition to black-box access) to grant auditors. We show that auditing can be cast as a natural hypothesis test, draw parallels hypothesis testing and legal procedure, and argue that this framing provides clear and interpretable guidance on audit implementation.Keywords
All Related Versions
This publication has 40 references indexed in Scilit:
- Causally estimating the effect of YouTube’s recommender system using counterfactual botsProceedings of the National Academy of Sciences, 2024
- Conformity assessment under the EU AI act general approachAI and Ethics, 2024
- Auditing YouTube’s recommendation system for ideologically congenial, extreme, and problematic recommendationsProceedings of the National Academy of Sciences, 2023
- The Anatomy of AI Audits2023
- Like-minded sources on Facebook are prevalent but not polarizingNature, 2023
- Wild patterns: Ten years after the rise of adversarial machine learningPattern Recognition, 2018
- Auditing black-box models for indirect influenceKnowledge and Information Systems, 2017
- Fair Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction InstrumentsBig Data, 2017
- Evasion Attacks against Machine Learning at Test Time2013
- Three naive Bayes approaches for discrimination-free classificationData Mining and Knowledge Discovery, 2010