Spatio-Temporal Graph Analytics on Secondary Affect Data for Improving Trustworthy Emotional AI
- 20 July 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 15 (1) , 30-49
Abstract
Ethical affective computing (AC) requires maximizing the benefits to users while minimizing its harm to obtain trust from users. This requires responsible development and deployment to ensure fairness, bias mitigation, privacy preservation, and accountability. To obtain this, we require methodologies that can quantify, visualize, analyze, and mine insights from affect data. Hence, in this paper, we propose a spatio-temporal model for representing secondary affect data from network sciences' perspective. We propose a network science-based model to represent spatio-temporal data, e.g., action units' sequences, and continuous affect reports. In particular, the proposed model captures the spatial and temporal strength of the relationship among essential variables in the data. The proposed model allows to analyze data as a whole system. We also demonstrated the use case of the model for graph analytics on secondary affect data that can assist to measure and quantify several issues that can be originated from the study setup, data recording devices, and the influences/biases that can originate from the perspective of the affect reporters. We also demonstrated the use cases of the proposed method on ethical trustworthy emotional AI via measuring biases from de-identified data and how it contributes towards ethics, transparency, value alignment, and governance.Keywords
Funding Information
- National Science Foundation (CNS-1629898)
This publication has 130 references indexed in Scilit:
- The Effect of Model Compression on Fairness in Facial Expression Recognition2023
- Counterfactual Fairness for Facial Expression Recognition2023
- Network-based prediction of COVID-19 epidemic spreading in ItalyApplied Network Science, 2020
- Investigating Bias and Fairness in Facial Expression Recognition2020
- A comprehensive survey on automatic facial action unit analysisThe Visual Computer, 2019
- Wired Emotions: Ethical Issues of Affective Brain–Computer InterfacesScience and Engineering Ethics, 2019
- Automated Pain Detection in Facial Videos of Children Using Human-Assisted Transfer Learning2019
- Deep Structure Inference Network for Facial Action Unit Recognition2018
- The many facets of community detection in complex networksApplied Network Science, 2017
- The Affective Triad: Stimuli, Questionnaires, and Measurements2011