Salman Sadiq Shuvo
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Salman Sadiq Shuvo has a total of 12 co-authors. Here are the top co-authors by number of shared publications:
- Yasin Yilmaz
Department of Electrical Engineering, University of South Florida, Tampa, FL, USA
5 shared publications - Rubel Ahmed
Department of Computer Science and Engineering, University of South Florida, Tampa, FL, USA
2 shared publications Tim Yoder
Pacific Northwest National Laboratory, Richland, WA, USA
2 shared publicationsYasin Yilmaz
2 shared publicationsAlan Bush
Honors College, University of South Florida, Tampa, FL, USA
1 shared publicationHasan Symum
University of South Florida, Tampa, FL, USA
1 shared publication
Latest Publications
- Open Access
- 28 August 2026
- Published by Saint Leo University in ICT Latin America
- Vol. 1 (1) , 10-23
Deep Reinforcement Learning for Residential EV Charging: A Comprehensive Survey
The rapid adoption of electric vehicles (EVs) introduces new challenges in managing residential charging demand, where uncoordinated behavior can increase energy costs, stress local grids, and reduce system efficiency. Deep Reinforcement Learning (DRL) has emerged as a powerful framework for developing intelligent, data-driven charging strategies that adapt to dynamic user behavior, time-varying electricity prices, and renewable generation. This paper presents a comprehensive survey of DRL techniques applied to home EV charging. We categorize existing research across three core dimensions—feature engineering, algorithm selection, and environment modeling—forming a unified analytical framework for evaluating and comparing DRL-based approaches. Key advancements in state representation, reward design, and control strategies are reviewed, alongside open challenges in scalability, interpretability, and generalization. Finally, the paper highlights future research opportunities including transformer-based forecasting, multi-objective DRL, and standardized simulation environments. The survey aims to guide researchers and practitioners toward developing robust, user-centric, and commercially viable DRL-based residential EV charging systems.
- 28 August 2026
- Published by Saint Leo University in ICT Latin America
- Vol. 1 (1) , 10-23
Open AccessDeep Reinforcement Learning for Residential EV Charging: A Comprehensive Survey
The rapid adoption of electric vehicles (EVs) introduces new challenges in managing residential charging demand, where uncoordinated behavior can increase energy costs, stress local grids, and reduce system efficiency. Deep Reinforcement Learning (DRL) has emerged as a powerful framework for developing intelligent, data-driven charging strategies that adapt to dynamic user behavior, time-varying electricity prices, and renewable generation. This paper presents a comprehensive survey of DRL techniques applied to home EV charging. We categorize existing research across three core dimensions—feature engineering, algorithm selection, and environment modeling—forming a unified analytical framework for evaluating and comparing DRL-based approaches. Key advancements in state representation, reward design, and control strategies are reviewed, alongside open challenges in scalability, interpretability, and generalization. Finally, the paper highlights future research opportunities including transformer-based forecasting, multi-objective DRL, and standardized simulation environments. The survey aims to guide researchers and practitioners toward developing robust, user-centric, and commercially viable DRL-based residential EV charging systems.
- Open Access
- 5 November 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in TechRxiv
Deep Reinforcement Learning for Residential EV Charging: A Comprehensive Survey
The rapid adoption of electric vehicles (EVs) introduces new challenges in managing residential charging demand, where uncoordinated behavior can increase energy costs, stress local grids, and reduce system efficiency. Deep Reinforcement Learning (DRL) has emerged as a powerful framework for developing intelligent, data-driven charging strategies that adapt to dynamic user behavior, time-varying electricity prices, and renewable generation. This paper presents a comprehensive survey of DRL techniques applied to home EV charging. We categorize existing research across three core dimensions-feature engineering, algorithm selection, and environment modeling-forming a unified analytical framework for evaluating and comparing DRLbased approaches. Key advancements in state representation, reward design, and control strategies are reviewed, alongside open challenges in scalability, interpretability, and generalization. Finally, the paper highlights future research opportunities including transformer-based forecasting, multi-objective DRL, and standardized simulation environments. The survey aims to guide researchers and practitioners toward developing robust, user-centric, and commercially viable DRL-based residential EV charging systems.
- 5 November 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in TechRxiv
Open AccessDeep Reinforcement Learning for Residential EV Charging: A Comprehensive Survey
The rapid adoption of electric vehicles (EVs) introduces new challenges in managing residential charging demand, where uncoordinated behavior can increase energy costs, stress local grids, and reduce system efficiency. Deep Reinforcement Learning (DRL) has emerged as a powerful framework for developing intelligent, data-driven charging strategies that adapt to dynamic user behavior, time-varying electricity prices, and renewable generation. This paper presents a comprehensive survey of DRL techniques applied to home EV charging. We categorize existing research across three core dimensions-feature engineering, algorithm selection, and environment modeling-forming a unified analytical framework for evaluating and comparing DRLbased approaches. Key advancements in state representation, reward design, and control strategies are reviewed, alongside open challenges in scalability, interpretability, and generalization. Finally, the paper highlights future research opportunities including transformer-based forecasting, multi-objective DRL, and standardized simulation environments. The survey aims to guide researchers and practitioners toward developing robust, user-centric, and commercially viable DRL-based residential EV charging systems.
- 10 January 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Smart Grid
- Vol. 14 (5) , 3889-3898
Demand-Side and Utility-Side Management Techniques for Increasing EV Charging Load
Electricity authorities need capacity assessment and expansion plans for efficiently charging the growing Electric vehicle (EV) fleet. Specifically, the distribution grid needs significant capacity expansion as it faces the most impact to accommodate the high variant residential EV charging load. Utility companies employ different scheduling policies for the maintenance of their distribution transformers (hereinafter, XFR). However, they lack scenario-based plans to cope with the varying EV penetration across locations and time. The contributions of this paper are twofold. First, we propose a customer feedback-based EV charging scheduling to simultaneously minimize the peak load for the distribution XFR and satisfy the customer needs. Second, we present a deep reinforcement learning (DRL) method for XFR maintenance, which focuses on the XFR’s effective age and loading to periodically choose the best candidate XFR for replacement. Our case study for a distribution feeder shows the adaptability and success of our EV load scheduling method in reducing the peak demand to extend the XFR life. Furthermore, our DRL-based XFR replacement policy outperforms the existing rule-based policies. Together, the two approaches provide a complete capacity planning tool for efficient XFR maintenance to cope with the increasing EV charging load.
- 10 January 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Smart Grid
- Vol. 14 (5) , 3889-3898
Demand-Side and Utility-Side Management Techniques for Increasing EV Charging Load
Electricity authorities need capacity assessment and expansion plans for efficiently charging the growing Electric vehicle (EV) fleet. Specifically, the distribution grid needs significant capacity expansion as it faces the most impact to accommodate the high variant residential EV charging load. Utility companies employ different scheduling policies for the maintenance of their distribution transformers (hereinafter, XFR). However, they lack scenario-based plans to cope with the varying EV penetration across locations and time. The contributions of this paper are twofold. First, we propose a customer feedback-based EV charging scheduling to simultaneously minimize the peak load for the distribution XFR and satisfy the customer needs. Second, we present a deep reinforcement learning (DRL) method for XFR maintenance, which focuses on the XFR’s effective age and loading to periodically choose the best candidate XFR for replacement. Our case study for a distribution feeder shows the adaptability and success of our EV load scheduling method in reducing the peak demand to extend the XFR life. Furthermore, our DRL-based XFR replacement policy outperforms the existing rule-based policies. Together, the two approaches provide a complete capacity planning tool for efficient XFR maintenance to cope with the increasing EV charging load.
- Open Access
- 11 July 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in TechRxiv
Demand-side and Utility-side Management Techniques for Increasing EV Charging Load
The source code will be updated in our lab github page https://github.com/Secure-and-Intelligent-SystemsLab/
- 11 July 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in TechRxiv
Open AccessDemand-side and Utility-side Management Techniques for Increasing EV Charging Load
The source code will be updated in our lab github page https://github.com/Secure-and-Intelligent-SystemsLab/
- 1 July 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Biomedical and Health Informatics
- Vol. 27 (6) , 2760-2770
Multi-Objective Reinforcement Learning Based Healthcare Expansion Planning Considering Pandemic Events
Hospital capacity expansion planning is critical for a healthcare authority, especially in regions with a growing diverse population. Policymaking to this end often requires satisfying two conflicting objectives, minimizing capacity expansion cost and minimizing the number of denial of service (DoS) for patients seeking hospital admission. The uncertainty in hospital demand, especially considering a pandemic event, makes expansion planning even more challenging. This work presents a multi-objective reinforcement learning (MORL) based solution for healthcare expansion planning to optimize expansion cost and DoS simultaneously for pandemic and non-pandemic scenarios. Importantly, our model provides a simple and intuitive way to set the balance between these two objectives by only determining their priority percentages, making it suitable across policymakers with different capabilities, preferences, and needs. Specifically, we propose a multi-objective adaptation of the popular Advantage Actor-Critic (A2C) algorithm to avoid forced conversion of DoS discomfort cost to a monetary cost. Our case study for the state of Florida illustrates the success of our MORL based approach compared to the existing benchmark policies, including a state-of-the-art deep RL policy that converts DoS to economic cost to optimize a single objective.
- 1 July 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Biomedical and Health Informatics
- Vol. 27 (6) , 2760-2770
Multi-Objective Reinforcement Learning Based Healthcare Expansion Planning Considering Pandemic Events
Hospital capacity expansion planning is critical for a healthcare authority, especially in regions with a growing diverse population. Policymaking to this end often requires satisfying two conflicting objectives, minimizing capacity expansion cost and minimizing the number of denial of service (DoS) for patients seeking hospital admission. The uncertainty in hospital demand, especially considering a pandemic event, makes expansion planning even more challenging. This work presents a multi-objective reinforcement learning (MORL) based solution for healthcare expansion planning to optimize expansion cost and DoS simultaneously for pandemic and non-pandemic scenarios. Importantly, our model provides a simple and intuitive way to set the balance between these two objectives by only determining their priority percentages, making it suitable across policymakers with different capabilities, preferences, and needs. Specifically, we propose a multi-objective adaptation of the popular Advantage Actor-Critic (A2C) algorithm to avoid forced conversion of DoS discomfort cost to a monetary cost. Our case study for the state of Florida illustrates the success of our MORL based approach compared to the existing benchmark policies, including a state-of-the-art deep RL policy that converts DoS to economic cost to optimize a single objective.
- 11 March 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Smart Grid
- Vol. 13 (4) , 2812-2821
Home Energy Recommendation System (HERS): A Deep Reinforcement Learning Method Based on Residents’ Feedback and Activity
Smart home appliances can take command and act intelligently, making them suitable for implementing optimization techniques. Artificial intelligence (AI) based control of these smart devices enables demand-side management (DSM) of electricity consumption. By integrating human feedback and activity in the decision process, this work proposes a deep Reinforcement Learning (RL) method for managing smart devices to optimize electricity cost and comfort residents. Our contributions are twofold. Firstly, we incorporate human feedback in the objective function of our DSM technique that we name Home Energy Recommendation System (HERS). Secondly, we include human activity data in the RL state definition to enhance the energy optimization performance. We perform comprehensive experimental analyses to compare the proposed deep RL approach with existing approaches that lack the aforementioned critical decision-making features. The proposed model is robust to varying resident activities and preferences and applicable to a broad spectrum of homes with different resident profiles.
- 11 March 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Smart Grid
- Vol. 13 (4) , 2812-2821
Home Energy Recommendation System (HERS): A Deep Reinforcement Learning Method Based on Residents’ Feedback and Activity
Smart home appliances can take command and act intelligently, making them suitable for implementing optimization techniques. Artificial intelligence (AI) based control of these smart devices enables demand-side management (DSM) of electricity consumption. By integrating human feedback and activity in the decision process, this work proposes a deep Reinforcement Learning (RL) method for managing smart devices to optimize electricity cost and comfort residents. Our contributions are twofold. Firstly, we incorporate human feedback in the objective function of our DSM technique that we name Home Energy Recommendation System (HERS). Secondly, we include human activity data in the RL state definition to enhance the energy optimization performance. We perform comprehensive experimental analyses to compare the proposed deep RL approach with existing approaches that lack the aforementioned critical decision-making features. The proposed model is robust to varying resident activities and preferences and applicable to a broad spectrum of homes with different resident profiles.
- Open Access
- 10 March 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in TechRxiv
Multi-Objective Reinforcement Learning Based Healthcare Expansion Planning Considering Pandemic Events
Hospital admission, Disease Burden (DB), and Social Vulnerability Index (SVI) data between 2010-2019 from State Inpatient Databases (SID) “HCUP State Inpatient Databases (SID). Healthcare Cost andUtilization Project (HCUP). 2010-2017.” [Online]. Available: www.hcup-us.ahrq.gov/sidoverview.jsp
- 10 March 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in TechRxiv
Open AccessMulti-Objective Reinforcement Learning Based Healthcare Expansion Planning Considering Pandemic Events
Hospital admission, Disease Burden (DB), and Social Vulnerability Index (SVI) data between 2010-2019 from State Inpatient Databases (SID) “HCUP State Inpatient Databases (SID). Healthcare Cost andUtilization Project (HCUP). 2010-2017.” [Online]. Available: www.hcup-us.ahrq.gov/sidoverview.jsp
- 1 November 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computational Social Systems
- Vol. 9 (4) , 1185-1196
Modeling and Simulating Adaptation Strategies Against Sea-Level Rise Using Multiagent Deep Reinforcement Learning
Sea-level rise (SLR) problem, which is a major outcome of climate change, has been well documented and studied. Although it is globally observed due to climate change, local projections are needed to plan SLR adaptation strategies accurately. Since SLR is a community-wide multistakeholder problem at the local level, adaptation strategies can be more successful if the main stakeholders, e.g., government, residents, and businesses, collaborate in shaping them. Simulating the local socioeconomic system around SLR, including the interactions between essential stakeholders and nature, can be an effective way of evaluating different adaptation strategies and planning the best strategy for the local community. This work presents how such an SLR socioeconomic system can be modeled as a Markov decision process (MDP) and simulated using multiagent reinforcement learning (RL). The proposed multiagent RL framework serves two purposes. It provides a general scenario planning tool to investigate the cost–benefit analysis of natural events (e.g., flooding and hurricane) and agents’ investments (e.g., infrastructure improvement). It also shows how much the total cost due to SLR can be reduced over time by optimizing the adaptation strategies. We demonstrate the proposed scenario planning tool using available economic data and sea-level projections for Pinellas County, Florida, in a case study.
- 1 November 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computational Social Systems
- Vol. 9 (4) , 1185-1196
Modeling and Simulating Adaptation Strategies Against Sea-Level Rise Using Multiagent Deep Reinforcement Learning
Sea-level rise (SLR) problem, which is a major outcome of climate change, has been well documented and studied. Although it is globally observed due to climate change, local projections are needed to plan SLR adaptation strategies accurately. Since SLR is a community-wide multistakeholder problem at the local level, adaptation strategies can be more successful if the main stakeholders, e.g., government, residents, and businesses, collaborate in shaping them. Simulating the local socioeconomic system around SLR, including the interactions between essential stakeholders and nature, can be an effective way of evaluating different adaptation strategies and planning the best strategy for the local community. This work presents how such an SLR socioeconomic system can be modeled as a Markov decision process (MDP) and simulated using multiagent reinforcement learning (RL). The proposed multiagent RL framework serves two purposes. It provides a general scenario planning tool to investigate the cost–benefit analysis of natural events (e.g., flooding and hurricane) and agents’ investments (e.g., infrastructure improvement). It also shows how much the total cost due to SLR can be reduced over time by optimizing the adaptation strategies. We demonstrate the proposed scenario planning tool using available economic data and sea-level projections for Pinellas County, Florida, in a case study.
- Open Access
Use of Machine Learning for Long Term Planning and Cost Minimization in Healthcare Management
The Healthcare system of a country is a crucial infrastructure that requires long-term capacity planning. The covid 19 outbreak pointed to the necessity of adequate hospital capacity, especially for developing countries like Bangladesh. The existing infrastructure planning of these countries emphasizes short-term goals and lacks vision planning for a long time horizon. It is in the country’s best interest to make long-term capacity expansion plans, a strategy the developed countries banked to provide adequate healthcare facilities to their residents. However, no single solution is appropriate for a different region. Hence, it is required to comprehensively study the situation and constraints of the specific region before providing expensive capacity expansion plans. This work focuses on applying a deep Reinforcement Learning based long-term hospital bed capacity expansion plan. We utilize the RNN-LSTM based population forecast, deep Reinforcement Learning (RL) based policy-making, and state-of-the-art Artificial Intelligence techniques to provide a solution. We perform a case study for the Abhaynagar Upazila of Jessore, one of the largest cities in the southwest part of Bangladesh, to analyze the benefits of such an approach compared to existing myopic policies. The experiment results show that the deep RL-based policy significantly minimizes cost over a 30-year expansion plan.
Open AccessUse of Machine Learning for Long Term Planning and Cost Minimization in Healthcare Management
The Healthcare system of a country is a crucial infrastructure that requires long-term capacity planning. The covid 19 outbreak pointed to the necessity of adequate hospital capacity, especially for developing countries like Bangladesh. The existing infrastructure planning of these countries emphasizes short-term goals and lacks vision planning for a long time horizon. It is in the country’s best interest to make long-term capacity expansion plans, a strategy the developed countries banked to provide adequate healthcare facilities to their residents. However, no single solution is appropriate for a different region. Hence, it is required to comprehensively study the situation and constraints of the specific region before providing expensive capacity expansion plans. This work focuses on applying a deep Reinforcement Learning based long-term hospital bed capacity expansion plan. We utilize the RNN-LSTM based population forecast, deep Reinforcement Learning (RL) based policy-making, and state-of-the-art Artificial Intelligence techniques to provide a solution. We perform a case study for the Abhaynagar Upazila of Jessore, one of the largest cities in the southwest part of Bangladesh, to analyze the benefits of such an approach compared to existing myopic policies. The experiment results show that the deep RL-based policy significantly minimizes cost over a 30-year expansion plan.
- Open Access
- 11 September 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in TechRxiv
Home Energy Recommendation System (HERS): A Deep Reinforcement Learning Method for Electricity Optimization in Smart Homes
Aras activity datasetNYISO dynamic electricity priceA2C implementation in PythonArticle under review in IEEE Transactions on Smart Grid
- 11 September 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in TechRxiv
Open AccessHome Energy Recommendation System (HERS): A Deep Reinforcement Learning Method for Electricity Optimization in Smart Homes
Aras activity datasetNYISO dynamic electricity priceA2C implementation in PythonArticle under review in IEEE Transactions on Smart Grid