Employing Hybrid AI Systems to Trace and Document Bias in ML Pipelines
Open Access
- 12 July 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 96821-96847
Abstract
Artificial Intelligence (AI) systems can introduce biases that lead to unreliable outcomes and, in the worst-case scenarios, perpetuate systemic and discriminatory results when deployed in the real world. While significant efforts have been made to create bias detection methods, developing reliable and comprehensive documentation artifacts also makes for valuable resources that address bias and aid in minimizing the harms associated with AI systems. Based on compositional design patterns, this paper introduces a documentation approach using a hybrid AI system to prompt the identification and traceability of bias in datasets and predictive AI models. To demonstrate the effectiveness of our approach, we instantiate our pattern in two implementations of a hybrid AI system. One follows an integrated approach and performs fine-grained tracing and documentation of the AI model. In contrast, the other hybrid system follows a principled approach and enables the documentation and comparison of bias in the input data and the predictions generated by the model. Through a use-case based on Fake News detection and an empirical evaluation, we show how biases detected during data ingestion steps (e.g., label, over-representation, activity bias) affect the training and predictions of the classification models. Concretely, we report a stark skewness in the distribution of input variables towards the Fake News label, we uncover how a predictive variable leads to more constraints in the learning process, and highlight open challenges of training models with unbalanced datasets. A video summarizing this work is available online ( https://youtu.be/v2GfIQPAy_4?si=BXtWOf97cLiZavyu ),and the implementation is publicly available on GitHub ( https://github.com/SDM-TIB/DocBiasKG ).Keywords
Funding Information
- Leibniz Association’s Open Access Publishing Fund
- European Union (EU)-Horizon 2020 Research and Innovation Program under the Marie Skłodowska-Curie Actions (MSCA) through the Project NoBias (860630)
- “Leibniz Best Minds: Program for Women Professors”
- “TrustKG-Transforming Data in Trustable Insights” Project (P99/2020)
This publication has 49 references indexed in Scilit:
- Policy advice and best practices on bias and fairness in AIEthics and Information Technology, 2024
- Neurosymbolic AI: the 3rd waveArtificial Intelligence Review, 2023
- Scaling up knowledge graph creation to large and heterogeneous data sourcesJournal of Web Semantics, 2022
- REVISE: A Tool for Measuring and Mitigating Bias in Visual DatasetsInternational Journal of Computer Vision, 2022
- Data distribution debugging in machine learning pipelinesThe VLDB Journal, 2022
- Modular design patterns for hybrid learning and reasoning systemsApplied Intelligence, 2021
- Neural, symbolic and neural-symbolic reasoning on knowledge graphsAI Open, 2021
- The FAIR Guiding Principles for scientific data management and stewardshipScientific Data, 2016
- Semantic Web in data mining and knowledge discovery: A comprehensive surveyJournal of Web Semantics, 2016
- Gene set enrichment analysis: A knowledge-based approach for interpreting genome-wide expression profilesProceedings of the National Academy of Sciences, 2005