Rashmi Naveen Raj

Personalized treatment is a paradigm shift in the medical field that provides individualized treatment strategies based on comprehensive patient data including habitat, genomic, and medical profile, etc. Integrating this into the conventional clinical process involves collaborative strategic planning with stakeholders from diverse sectors. The scope of this survey is limited to studies that apply reinforcement learning, deep reinforcement learning, and multi-objective deep reinforcement learning to sequential decision making in personalized treatment. The review addresses these main questions: Why are reinforcement learning and its variants suitable for personalized treatment?How are these algorithms applied in clinical settings of various domains? How are states, actions and rewards defined? What are the requirements, challenges at every stage of model development for deploying these models in safety-critical healthcare applications? Early studies primarily focused on reinforcement learning with single objective, while more works have increasingly adopted deep variants to manage high-dimensional state-action spaces, and a sparse literature exists with multi-objective approach to address conflicting treatment options. Most existing research is concentrated on using publicly available MIMIC- III /IV dataset in ICU settings, oncology related resources including, The Cancer Genome Atlas, Genomics of Drug Sensitivity in Cancer, The Catalogue of Somatic Mutations In Cancer. To the best of our knowledge, this is the first survey to emphasize: (i) the need for multi-objective optimization with sequential personalized treatment planning, (ii) the integration of non-clinical objectives such as patient preferences, hospital infrastructure, insurance coverage, etc., and stake-holders into decision framework, (iii) the theoretical and conceptual model of a multi-modal multi-objective deep reinforcement learning aiming at optimizing the treatment regimes using a newly introduced time-weighted reward integration block to balance multiple, potentially conflicting clinical as well as non-clinical objectives.

Accurate and early identification of ocular diseases is essential to prevent vision impairment and enable timely medical intervention. In routine clinical practice, ophthalmologists rely on a structured diagnostic workflow that incorporates multiple imaging modalities to manually assess and diagnose ocular diseases. However, interpreting each modality requires significant clinical experience and can be time-consuming. These limitations can be effectively addressed through the application of AI (Artificial intelligence)-driven multimodal fusion techniques. In this study, we conducted an empirical investigation to assess the impact of different fusion strategies—including early, intermediate, and late fusion—on diagnostic performance, training requirements, and interpretability. The proposed methodology was evaluated using three publicly available datasets: FFA-Fundus (Fundus fluorescein angiography), GAMMA (Glaucoma Analysis and Multi-Modal Assessment), and OLIVES (Ophthalmic Labels to Investigate Visual Eye Semantics). Experimental results demonstrate that multimodal feature fusion improves disease detection performance. Although fused models typically required an increase in training parameters compared to single-modality models, they provided interpretability on par with that of individual single-modal networks. However, inference time increased by approximately 50% for multimodal architectures. These findings underscore the value of integrating diverse ophthalmic imaging modalities to enhance diagnostic accuracy in automated disease detection systems. At the same time, the results highlight that unimodal models containing highly discriminative features can also perform competitively, particularly when a single modality is sufficient for disease identification. Multimodal fusion provides the greatest benefit in scenarios where complementary information across modalities contributes distinct and non-redundant features. Furthermore, fusing all available modalities may not be optimal due to increased computational cost and reduced inference efficiency; thus, selective modality integration and lightweight fusion strategies are essential to balance accuracy, interpretability, and efficiency in clinical deployment.

Autonomous vehicles mitigate road accidents and provide safe transportation with a smooth traffic flow. They are expected to greatly improve the quality of the elderly or people with impairments by improving their mobility due to the ease of access to transportation. Autonomous vehicles sense the driving environment and navigate through it without human intervention. And, Deep Reinforcement Learning (DRL) is witnessed as a powerful machine learning solution to address a sequential decision problem in autonomous vehicles. The detailed state-of-the-art work in autonomous vehicles using DRL algorithms along with future research directions is discussed. Due to the high dimensional action space, two continuous action space DRL algorithms: Deterministic Policy Gradient (DDPG) and Proximal Policy Optimization (PPO) are chosen to address the complex autonomous driving problem. The proposed DDPG and PPO based decision-making models are trained and tested using the TORC simulator. Both the algorithms are trained for the same number of episodes for lane keeping as well as multi-agent collision avoidance scenarios. To the best of our knowledge, this is the first paper to present the comparative performance analysis of these two algorithms, and DDPG is found to perform better in terms of higher reward and faster convergence than PPO. Hence, DDPG is a suitable option in the design of a decision model for autonomous driving.

Numerous accidents and fatalities occur every year across the world as a result of the reckless driving of drivers and the ever-increasing number of vehicles on the road. Due to these factors, autonomous cars have attracted enormous attention as a potentially game-changing technology to address a number of persistent problems in the transportation industry. Autonomous vehicles need to be modeled as intelligent agents with the capacity to observe, and perceive the complex and dynamic environment on the road, and decide an action with the highest priority to the lives of people in every scenarios. The proposed deep deterministic policy gradient-based sequential decision algorithm models the autonomous vehicle as a learning agent and trains it to drive on a lane, overtake a static and a moving vehicle, and avoid collisions with obstacles on the front and right side. The proposed work is simulated using a TORC simulator and has shown the expected performance under the above-said scenarios.

Sepsis is a lethal infection-related illness that has an extremely high fatality rate, especially among intensive care unit patients. Early and precise recognition of sepsis is critical as delayed treatment increases the mortality rate dramatically. System inflammatory response syndrome, quick sequential organ failure assessment, and modified early warning score are the traditional clinical score systems in practice to detect sepsis. But the scoring systems fail in the early prediction of sepsis, a stage in which if a patient is treated immediately, the mortality rate will reduce significantly. The proposed classifier can accurately predict sepsis up to six hours before the disease is clinically diagnosed. The patient’s electronic medical records, demographics, and vital signs are used to achieve this. The study uses data set adaptive data preprocessing strategies. The proposed method adds value to existing literature by introducing a novel outlier-based mean-median data imputation technique that enhances the prediction’s overall accuracy. The primary factors that influence the classifier’s predictions have been outlined, making the model easier to understand for medical professionals. For the classification of patients as sepsis positive or negative, four algorithms were investigated: Random Forest, Logistic Regression, Gradient Boosting, and Decision Tree. Of all the prediction algorithms, Random Forest gives the best results with an accuracy of 99.01%, F1-score of 99%, and an area under the receiver operator characteristic curve of 99.99%. Even for a 24-hour early prediction of sepsis, the random forest method is proven to provide greater prediction accuracy while logistic regression provides the least prediction accuracy. We attribute this to the fact that, unlike regression models, random forests do not require that the model have a linear relationship between the dependent and independent variables. The evaluation measures produced are useful and can be tremendously valuable in predicting sepsis in a timely and accurate manner.

Network Simulator(NS-2) is an open source discrete event simulator, widely used by the networking research community. NS-2 provides flexibility for the researcher to extend the simulator functionality for any new area of research in networks. The primary objective of the paper is to investigate the development and simulation of the interface assignment in multi-interface multi-channel wireless adhoc network in NS-2. Accordingly, existing ad hoc on-demand distance vector(AODV) routing protocol which was designed for single interface single channel (S-AODV) was studied and modified to support multi-interface multi-channel (M-AODV)configuration. Simulation results show better performance of M-AODV over S-AODV.