Alfonso Cannavale

Context. Fairness of machine learning (ML) systems is increasingly recognized as a critical non-functional requirement in software engineering. In modern ML-enabled systems, where data evolves continuously, and models are frequently retrained, current practices remain fundamentally reactive: fairness is assessed only after training, and violations caused by data drift are detected too late, triggering costly retraining cycles. Objective. We introduce FADE (Fairness Assessment under Dataset Evolution), a framework for preventive fairness stress testing of ML datasets. FADE assesses the structural robustness of a dataset by applying mutation-based perturbations to individual features and measuring their effect on data-level bias symptoms. Features whose fairness profile degrades under small changes are identified as fairness-fragile: latent vulnerabilities often residing in non-sensitive attributes that serve as proxies for discrimination. Method. We validate FADE through a two-phase empirical study on four benchmark datasets spanning socio-economic, criminal justice, healthcare, and financial domains. Results. FADE identified 30 high-risk configurations involving 21 unique fairness-fragile features, 76% of which are non-sensitive. In 93.3% of configurations, data-level fragilities translated into statistically significant fairness variations in downstream models, consistent across Random Forest and XGBoost. We also observe that perturbing a fragile feature can reduce rather than amplify model bias, revealing how deeply certain features are embedded in discriminatory behavior. Conclusion. FADE enables teams to surface fairness risks before model training, supporting a shift from reactive post-hoc assessment toward proactive data quality assurance in ML-enabled systems.

Test flakiness arises when a test case exhibits inconsistent behavior by alternating between passing and failing states when executed against the same code. Previous research showed the significance of the problem in practice, proposing empirical studies into the nature of flakiness and automated techniques for its detection. Machine learning models emerged as a promising approach for flaky test prediction. However, existing research has predominantly focused on within-project scenarios, where models are trained and tested using data from a single project. On the contrary, little is known about how flaky test prediction models may be adapted to software projects lacking sufficient historical data for effective prediction. In this paper, we address this gap by proposing a large-scale assessment of flaky test prediction in cross-project scenarios, i.e., in situations where predictive models are trained using data coming from external projects. Leveraging a dataset of 1,385 flaky tests from 29 open-source projects, we examine static test flakiness prediction models and evaluate feature- and instance-based filtering methods for cross-project predictions. Our study underscores the difficulties in utilizing cross-project flaky test data and underscores the significance of filtering methods in enhancing prediction accuracy. Notably, we find that the TrAdaBoost filtering method significantly reduces data heterogeneity, leading to an F-Measure of 70%.