Oussama Kerdjidj

Augmented Reality (AR) technology offers promising applications in healthcare by enabling interactive 3D visualization of anatomical structures. However, current AR implementations often lack patient-specific detail, limiting their effectiveness in clinical settings. In this paper, we present BrainAR, an innovative mobile AR-based application designed for the automatic segmentation, 3D visualization, localization, and interaction with brain tumors using multiparametric 3D Magnetic Resonance Imaging (MRI) data. Our method leverages a 3D Residual U-Net, trained on the BraTS2021 dataset, achieving a mean Dice score of 0.886 for accurate tumor segmentation. The segmentation outputs are integrated into a real-time 3D engine to enable precise and dynamic visualization of brain tumors. Key contributions of our work include: 1) a server-side deployment of the segmentation model for online, patient-specific inference; 2) seamless AR integration enabling interactive exploration through hand gestures and voice commands; and 3) a mobile-based platform aimed at enhancing accessibility and usability in clinical environments. The proposed solution facilitates early detection and diagnosis by providing clinicians with an intuitive, immersive, and patient-specific tool for enhanced medical imaging interaction.

Cardiovascular diseases (CVDs) continue to be a leading cause of mortality globally, highlighting the urgent need for timely and accurate diagnosis. Electrocardiography (ECG) is a vital diagnostic tool for detecting and monitoring various heart conditions by analysing the heart’s electrical activity; however, manually identifying ECG features and classifying heartbeats is a complex and time-consuming process that demands significant expertise. To address this challenge, we have developed ArythmiAR, a novel system that integrates Convolutional Neural Networks (CNN) with Augmented Reality (AR) to enable interactive diagnosis with 3D visualisation and real-time engagement. ArythmiAR offers several key innovations: deep learning-based ECG classification for precise arrhythmia detection, 3D heart modelling and assembly for detailed visualisation, an AR interface for deploying CNN models, 3D localisation of heart sub-regions responsible for arrhythmia anomalies, and enhanced 3D visualisation and interaction capabilities. Our study explores various ECG classification techniques, employing data rebalancing strategies to enhance model performance, with a particular focus on Multilayer Perceptron (MLP) and CNN models, which demonstrated highly competitive results on the PhysioNet MIT-BIH Arrhythmia dataset, achieving an accuracy of 99.07% with the MLP model. Remarkably, this work also involves deploying the ECG classification deep learning model within an AR environment, presenting a prototype for augmented rendering that allows users to localise, visualise, and interact with specific heart regions responsible for arrhythmias. This platform empowers medical professionals to make more accurate diagnoses and develop effective treatment strategies, thereby improving overall patient care.

Augmented reality (AR) technology is rapidly advancing, enabling interactive 3D virtual environments and fostering human interaction and participation. AR has made significant strides in medicine, particularly in medical imaging, leading to improved diagnostic capabilities. To contribute to this transformative landscape, we have developed an accessible and cost-effective AR application that can be used with affordable smartphones. Our application offers automatic brain tumour segmentation, 6 degrees of freedom (6 DoF) phantom head pose estimation, AR visualisation, and interaction, eliminating the need for expensive head-mounted devices (HMD). This paper introduces a novel approach that utilises transfer learning and fine-tuning techniques to train a deep learning model capable of accurately estimating the 6 DoF object pose using RGB images alone. We also present a brain tumour segmentation method based on a specific technique of Gradient Vector Flow (GVF)-based active contour models and demonstrate 3D reconstruction and printing of a phantom head. Furthermore, we propose a comprehensive mobile AR platform, ARBrain, specifically designed for localisation and visualisation of brain tumours. This platform incorporates the deployment of the 6 DoF pose estimator into the AR mobile platform, enabling 3D AR visualisation of the brain and tumour, as well as AR interaction through voice and hand-tactile gesture commands. To evaluate the effectiveness of our approach, we employ a layering procedure and compare it with other recent virtual overlay techniques. Through automatic tumour segmentation, precise pose estimation, and advanced AR interaction capabilities, our ARBrain platform offers an accessible and cost-effective solution for brain tumour localisation and visualisation, empowering medical professionals to make accurate diagnoses and treatment plans.

Indoor localization (IL) is a significant topic of study with several practical applications, particularly in the context of the Internet of Things (IoT) and smart cities. The area of IL has evolved greatly in recent years due to the introduction of numerous technologies such as WiFi, Bluetooth, cameras, and other sensors. Despite the growing interest in this field, there are numerous challenges and drawbacks that must be addressed to develop more accurate and sustainable systems for IL. This review study gives an in-depth look into IL, covering the most promising artificial intelligence-based and hybrid strategies that have shown excellent potential in overcoming some of the limitations of classic methods within IoT environments. In addition, the paper investigates the significance of high-quality datasets and evaluation metrics in the design and assessment of IL algorithms. Furthermore, this overview study emphasizes the crucial role that machine learning techniques, such as deep learning and transfer learning, play in the advancement of IL. A focus on the importance of IL and the various technologies, methods, and techniques that are being used to improve it. Finally, the survey highlights the need for continued research and development to create more accurate and scalable techniques that can be applied across a range of IoT-related industries, such as evacuation-egress routes, hazard-crime detection, smart occupancy-driven energy reduction and asset tracking and management.

Indoor localization systems predominantly depend on one-dimensional signal measurements, such as the Received Signal Strength Indication (RSSI) from Bluetooth or WiFi access points (AP). Such methods, however, grapple with issues like interference from other APs and environmental challenges. To address these, our paper introduces an innovative indoor localization technique employing a classification system bolstered by transfer learning. Instead of relying solely on one-dimensional signals, we transform them into images using techniques like spectrograms, scalograms, or Gramian Angular Fields. These transformed images feed into our classification system using a transfer learning approach. We tested our method on two public datasets, achieving remarkable accuracy rates of 99% with the Google-Net model and 98% with the Squeeze-Net architecture. These figures underscore the efficacy of our technique for indoor localization, marking a notable advancement over existing strategies.