Masoud Abedi

Background: Magnetic resonance imaging (MRI) is commonly used for analyzing white matter abnormalities in the human brain. Integrating machine learning into MRI analysis can enhance diagnostic processes. However, the application of such techniques for white matter analysis in clinical practice is often limited when MRI data are multi-scanner (i.e., heterogeneous), particularly in scenarios with limited data, as seen in rare diseases. Therefore, it is crucial to develop methods that are highly independent of the MRI scanner and acquisition protocol. Results: This study introduces HeteroMRI, a deep learning method for classifying MRIs based on white matter abnormalities. Most importantly, HeteroMRI mitigates the effects of data heterogeneity on classification performance. Herein, HeteroMRI is employed to detect brain MRIs with white matter abnormalities. This method utilizes intensity clustering of the white matter tissue to reduce the effects of the heterogeneity of MRIs. MRI data from 11 public datasets with 40 MRI protocols are included. By using 200 MRIs for training the model, the binary classifier achieves an average accuracy of 93% ± 4%. Furthermore, the method is evaluated in limited data scenarios, simulating conditions of rare diseases. By reducing the data by 64% and 75%, the model’s accuracy has a 4% and 12% decrease, respectively. Conclusions: The presented method opens new avenues for white matter abnormality-related classification of heterogeneous MRI data without additional machine learning methods to reduce MRI heterogeneity. This classification approach demonstrates a high degree of independence from the MRI scanner and protocol, while also proving to be relatively generalizable to unseen MRI protocols.

Machine learning (ML) has emerged as a vital asset for researchers to analyze and extract valuable information from complex datasets. However, developing an effective and robust ML pipeline can present a real challenge, demanding considerable time and effort, thereby impeding research progress. Existing tools in this landscape require a profound understanding of ML principles and programming skills. Furthermore, users are required to engage in the comprehensive configuration of their ML pipeline to obtain optimal performance. To address these challenges, we have developed a novel tool called Machine Learning Made Easy (MLme) that streamlines the use of ML in research, specifically focusing on classification problems at present. By integrating 4 essential functionalities—namely, Data Exploration, AutoML, CustomML, and Visualization—MLme fulfills the diverse requirements of researchers while eliminating the need for extensive coding efforts. To demonstrate the applicability of MLme, we conducted rigorous testing on 6 distinct datasets, each presenting unique characteristics and challenges. Our results consistently showed promising performance across different datasets, reaffirming the versatility and effectiveness of the tool. Additionally, by utilizing MLme’s feature selection functionality, we successfully identified significant markers for CD8⁺ naive (BACH2), CD16⁺ (CD16), and CD14⁺ (VCAN) cell populations. MLme serves as a valuable resource for leveraging ML to facilitate insightful data analysis and enhance research outcomes, while alleviating concerns related to complex coding scripts. The source code and a detailed tutorial for MLme are available at https://github.com/FunctionalUrology/MLme.

Background: In recent years, three-dimensional (3D) spheroid models have become increasingly popular in scientific research as they provide a more physiologically relevant microenvironment that mimics in vivo conditions. The use of 3D spheroid assays has proven to be advantageous as it offers a better understanding of the cellular behavior, drug efficacy, and toxicity as compared to traditional two-dimensional cell culture methods. However, the use of 3D spheroid assays is impeded by the absence of automated and user-friendly tools for spheroid image analysis, which adversely affects the reproducibility and throughput of these assays. Results: To address these issues, we have developed a fully automated, web-based tool called SpheroScan, which uses the deep learning framework called Mask Regions with Convolutional Neural Networks (R-CNN) for image detection and segmentation. To develop a deep learning model that could be applied to spheroid images from a range of experimental conditions, we trained the model using spheroid images captured using IncuCyte Live-Cell Analysis System and a conventional microscope. Performance evaluation of the trained model using validation and test datasets shows promising results. Conclusion: SpheroScan allows for easy analysis of large numbers of images and provides interactive visualization features for a more in-depth understanding of the data. Our tool represents a significant advancement in the analysis of spheroid images and will facilitate the widespread adoption of 3D spheroid models in scientific research. The source code and a detailed tutorial for SpheroScan are available at https://github.com/FunctionalUrology/SpheroScan . Key Points: A deep learning model was trained to detect and segment spheroids in images from microscopes and Incucytes. The model performed well on both types of images with the total loss decreasing significantly during the training process. A web tool called SpheroScan was developed to facilitate the analysis of spheroid images, which includes prediction and visualization modules. SpheroScan is efficient and scalable, making it possible to handle large datasets with ease. SpheroScan is user-friendly and accessible to researchers, making it a valuable resource for the analysis of spheroid image data.

In recent years, 3-dimensional (3D) spheroid models have become increasingly popular in scientific research as they provide a more physiologically relevant microenvironment that mimics in vivo conditions. The use of 3D spheroid assays has proven to be advantageous as it offers a better understanding of the cellular behavior, drug efficacy, and toxicity as compared to traditional 2-dimensional cell culture methods. However, the use of 3D spheroid assays is impeded by the absence of automated and user-friendly tools for spheroid image analysis, which adversely affects the reproducibility and throughput of these assays. To address these issues, we have developed a fully automated, web-based tool called SpheroScan, which uses the deep learning framework called Mask Regions with Convolutional Neural Networks (R-CNN) for image detection and segmentation. To develop a deep learning model that could be applied to spheroid images from a range of experimental conditions, we trained the model using spheroid images captured using IncuCyte Live-Cell Analysis System and a conventional microscope. Performance evaluation of the trained model using validation and test datasets shows promising results. SpheroScan allows for easy analysis of large numbers of images and provides interactive visualization features for a more in-depth understanding of the data. Our tool represents a significant advancement in the analysis of spheroid images and will facilitate the widespread adoption of 3D spheroid models in scientific research. The source code and a detailed tutorial for SpheroScan are available at https://github.com/FunctionalUrology/SpheroScan.

Background: Assessing the performance of machine learning (ML) models requires careful consideration of the evaluation metrics used. It is often necessary to utilize multiple metrics to gain a comprehensive understanding of a trained model’s performance, as each metric focuses on a specific aspect. However, comparing the scores of these individual metrics for each model to determine the best-performing model can be time-consuming and susceptible to subjective user preferences, potentially introducing bias. Results: We propose the Machine Learning Cumulative Performance Score (MLcps), a novel evaluation metric for classification problems. MLcps integrates several precomputed evaluation metrics into a unified score, enabling a comprehensive assessment of the trained model’s strengths and weaknesses. We tested MLcps on 4 publicly available datasets, and the results demonstrate that MLcps provides a holistic evaluation of the model’s robustness, ensuring a thorough understanding of its overall performance. Conclusions: By utilizing MLcps, researchers and practitioners no longer need to individually examine and compare multiple metrics to identify the best-performing models. Instead, they can rely on a single MLcps value to assess the overall performance of their ML models. This streamlined evaluation process saves valuable time and effort, enhancing the efficiency of model evaluation. MLcps is available as a Python package at https://pypi.org/project/MLcps/.

Motivation A performance metric is a tool to measure the correctness of a trained Machine Learning (ML) model. Numerous performance metrics have been developed for classification problems making it overwhelming to select the appropriate one since each of them represents a particular aspect of the model. Furthermore, selection of a performance metric becomes harder for problems with imbalanced and/or small datasets. Therefore, in clinical studies where datasets are frequently imbalanced and, in situations when the prevalence of a disease is low or the collection of patient samples is difficult, deciding on a suitable metric for performance evaluation of an ML model becomes quite challenging. The most common approach to address this problem is measuring multiple metrics and compare them to identify the best-performing ML model. However, comparison of multiple metrics is laborious and prone to user preference bias. Furthermore, evaluation metrics are also required by ML model optimization techniques such as hyperparameter tuning, where we train many models, each with different parameters, and compare their performances to identify the best-performing parameters. In such situations, it becomes almost impossible to assess different models by comparing multiple metrics. Results Here, we propose a new metric called Machine Learning Cumulative Performance Score (MLcps) as a Python package for classification problems. MLcps combines multiple pre-computed performance metrics into one metric that conserves the essence of all pre-computed metrics for a particular model. We tested MLcps on 4 different publicly available biological datasets and the results reveal that it provides a comprehensive picture of overall model robustness. Availability MLcps is available at https://pypi.org/project/MLcps/ and cases of use are available at https://mybinder.org/v2/gh/FunctionalUrology/MLcps.git/main. Supplementary information Supplementary data are available at Bioinformatics online.

Modern machine and deep learning methods require large datasets to achieve reliable and robust results. This requirement is often difficult to meet in the medical field, due to data sharing limitations imposed by privacy regulations or the presence of a small number of patients (e.g., rare diseases). To address this data scarcity and to improve the situation, novel generative models such as Generative Adversarial Networks (GANs) have been widely used to generate synthetic data that mimic real data by representing features that reflect health-related information without reference to real patients. In this paper, we consider several GAN models to generate synthetic data used for training binary (malignant/benign) classifiers, and compare their performances in terms of classification accuracy with cases where only real data are considered. We aim to investigate how synthetic data can improve classification accuracy, especially when a small amount of data is available. To this end, we have developed and implemented an evaluation framework where binary classifiers are trained on extended datasets containing both real and synthetic data. The results show improved accuracy for classifiers trained with generated data from more advanced GAN models, even when limited amounts of original data are available.

The constant upward movement of data-driven medicine as a valuable option to enhance daily clinical practice has brought new challenges for data analysts to get access to valuable but sensitive data due to privacy considerations. One solution for most of these challenges are Distributed Analytics (DA) infrastructures, which are technologies fostering collaborations between healthcare institutions by establishing a privacy-preserving network for data sharing. However, in order to participate in such a network, a lot of technical and administrative prerequisites have to be made, which could pose bottlenecks and new obstacles for non-technical personnel during their deployment. We have identified three major problems in the current state-of-the-art. Namely, the missing compliance with FAIR data principles, the automation of processes, and the installation. In this work, we present a seamless on-boarding workflow based on a DA reference architecture for data sharing institutions to address these problems. The on-boarding service manages all technical configurations and necessities to reduce the deployment time. Our aim is to use well-established and conventional technologies to gain acceptance through enhanced ease of use. We evaluate our development with six institutions across Germany by conducting a DA study with open-source breast cancer data, which represents the second contribution of this work. We find that our on-boarding solution lowers technical barriers and efficiently deploys all necessary components and is, therefore, indeed an enabler for collaborative data sharing.