Per Erik Strandberg

Artificial Intelligence (AI) is widely used in Industry 4.0 and Industry 5.0 applications for anomaly detection and predictive analytics. However, the opaque decision-making of most Machine Learning (ML) models limits interpretability and hinders effective decision-making and root cause analysis. Explainable AI (XAI) seeks to address these challenges; however, the combined processes of anomaly detection and explanation generation often require substantial domain expertise, limiting their accessibility to a broader audience. To address this gap, this paper presents EXACT (EXplainable Anomaly Classification Tool), a novel, modular software framework that enables practitioners and researchers to rapidly and easily perform explainable anomaly detection on time-series data, regardless of domain expertise. EXACT integrates time-series anomaly detection with XAI techniques into a single end-to-end workflow, including data handling, anomaly injection, parameter tuning, model training, anomaly detection, explanation generation, and visualization, making these features broadly accessible and user-friendly to the wider community. The effectiveness of EXACT is demonstrated through rigorous evaluations conducted on three public time-series datasets from distinct domains, using three anomaly detection models and three widely adopted XAI methods. Among these, XGBoost exhibits the most balanced predictive performance, achieving 99.96% accuracy, 82.61% F1-score, and 98.59% ROC-AUC on the Credit Card Fraud dataset, while Decision Trees demonstrate significantly lower training times. In terms of explainability, SHAP consistently delivers high-quality feature rankings, reaching an NDCG score of 0.98 on the Credit Card Fraud dataset, whereas LIME provides substantially faster explanation generation. These results demonstrate that EXACT effectively facilitates explainable anomaly detection in an efficient, user-friendly manner, while also providing a systematic analysis of trade-offs among predictive performance, interpretability, and computational efficiency. EXACT is publicly available at GitHub (https://github.com/TedBoman/EXACT), and the community is invited to contribute to its continued development and extension.

Cyber-physical systems (CPSs) are complex systems that integrate physical, computational, and communication subsystems. The heterogeneous nature of these systems makes their safety assurance challenging. In this paper, we propose a novel automated approach for guardrailing cyber-physical systems using property-based tests (PBTs) generated by Large Language Models (LLMs). Our approach employs an LLM to extract properties from the code and documentation of CPSs. Next, we use the LLM to generate PBTs that verify the extracted properties on the CPS. The generated PBTs have two uses. First, they are used to test the CPS before it is deployed, i.e., at design time. Secondly, these PBTs can be used after deployment, i.e., at run time, to monitor the behavior of the system and guardrail it against unsafe states. We implement our approach in ChekProp and conduct preliminary experiments to evaluate the generated PBTs in terms of their relevance (how well they match manually crafted properties), executability (how many run with minimal manual modification), and effectiveness (coverage of the input space partitions). The results of our experiments and evaluation demonstrate a promising path forward for creating guardrails for CPSs using LLM-generated property-based tests.

This study addresses the intersection of ethics and software test automation, driven by the use of Artificial Intelligence (AI) in software testing. Complex ethical challenges arise while applying AI and automation in the different stages of software testing such as case creation and selection, test execution, resource allocation, and interpretation of test results thereafter utilised in the industry. Despite the availability of ethical guidelines, the practical implementation of ethical AI in test automation needs to be clarified. To this end, we identified nine key ethical themes: human control and responsibility, justice and fairness, explainability (including logging and monitoring), human-centric risk analysis, stakeholder involvement, impact on human relations, technical risk analysis, sustainability and waste, and privacy. These themes were examined through a questionnaire, iteratively refined and evaluated through a pilot study with four experts, followed by a survey involving 54 participants from diverse backgrounds in software test automation. The findings reveal an agreement among participants on the importance of issues like explainability, privacy, and technical risks while highlighting the lack of consideration of human-centric risks. This paper brings some insights into AI ethics in software testing, specifically proposing the adoption of the questionnaire as guidance for professionals and researchers dealing with the ethical challenges of AI-powered test automation.

Anomaly detection in the monitoring systems of DevOps environments is crucial for ensuring system reliability, preventing downtime, and maintaining the efficiency of continuous integration and continuous deployment pipelines. Artificial Intelligence (AI)-based solutions for automated anomaly detection in DevOps workflows are attracting growing research interest. However, challenges remain regarding the lack of ground truth data from DevOps systems, as well as difficulties in storing, processing, and visualizing the collected data. Furthermore, most of these datasets are unlabeled, making it unclear what constitutes anomalous system behavior, and no generalized approach exists for selecting the most suitable AI algorithms for anomaly detection in such contexts. To address these challenges, this paper publishes a comprehensive time series dataset from 19 different DevOps test systems, comprising 24 performance metrics sampled twice per minute over 30 days. Moreover, to benchmark the dataset, we first label a subset of the dataset based on feedback from the DevOps experts within the industry context. Then six different algorithms are employed on the dataset and their performance in anomaly detection is evaluated using three different Area Under the Curve (AUC) metrics. Additionally, this paper presents a tool for storing, visualizing, monitoring, and integrating anomaly detection algorithms and makes it available to the community. The performance evaluation results aiming to establish a baseline for further research show that AI-based anomaly detection can significantly benefit DevOps workflows but emphasize the need for algorithm selection and parameter tuning tailored to the specific industry context and dataset.

Effective data management and sharing are critical success factors in industry-academia collaboration. This paper explores the motivations and lessons learned from publishing open data sets in such collaborations. Through a survey of participants in a European research project that published 13 data sets, and an analysis of metadata from almost 281 thousand datasets in Zenodo, we collected qualitative and quantitative results on motivations, achievements, research questions, licences and file types. Through inductive reasoning and statistical analysis we found that planning the data collection is essential, and that only few datasets (2.4%) had accompanying scripts for improved reuse. We also found that authors are not well aware of the importance of licences or which licence to choose. Finally, we found that data with a synthetic origin, collected with simulations and potentially mixed with real measurements, can be very meaningful, as predicted by Gartner and illustrated by many datasets collected in our research project.

Regression testing in software development checks if new software features affect existing ones. Regression testing is a key task in continuous development and integration, where software is built in small increments and new features are integrated as soon as possible. It is therefore important that developers are notified about possible faults quickly. In this article, we propose a test case prioritization schema that combines the use of a static and a dynamic prioritization algorithm. The dynamic prioritization algorithm rearranges the order of execution of tests on the fly, while the tests are being executed. We propose to use a conditional probability dynamic algorithm for this. We evaluate our solution on three industrial datasets and utilize Average Percentage of Fault Detection for that. The main findings are that our dynamic prioritization algorithm can: a) be applied with any static algorithm that assigns a priority score to each test case b) can improve the performance of the static algorithm if there are failure correlations between test cases c) can also reduce the performance of the static algorithm, but only when the static scheduling is performed at a near optimal level.

We need to find an effective prioritization of regression test cases due to their growing number. This may happen on parallel test systems and software branches. We compared regression test prioritization approaches against several goals of importance in an industrial context. We experimentally compared different simulated annealing approaches, hypothetical ideal and worst prioritizations, as well as reference prioritizations such as random, historical failure rate, age, etc. These were evaluated against a heuristic metric that combines several factors, as well as reference metrics such as failure count, days since last execution, etc. By simulating resource starvation in terms of available time, we found that some approaches rapidly degraded, e.g., by only prioritizing recently failed tests, the average number of nights since last execution was about five times as bad as for a random selection. The simulated annealing approach with large search space and many iterations came out best for many metrics. Interestingly, the poorest prioritization was achieved by aiming at diversity, and the coverage-based prioritization was poor at finding failures.