Yogesh Barve
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Yogesh Barve has a total of 37 co-authors. Here are the top co-authors by number of shared publications:
Aniruddha Gokhale
Vanderbilt University,Department of Electrical Engineering and Computer Science,Nashville,TN,USA,37212
12 shared publications- Anirban Bhattacharjee
Vanderbilt University
7 shared publications Shweta Khare
Vanderbilt University,EECS Dept,Nashville,TN,USA
5 shared publicationsHongyang Sun
Vanderbilt University,Department of Electrical Engineering and Computer Science,Nashville,TN,USA,37212
4 shared publicationsShashank Shekhar
4 shared publicationsZhuangwei Kang
Vanderbilt University,Department of Electrical Engineering and Computer Science,Nashville,TN,USA,37212
3 shared publications
Latest Publications
- 31 March 2023
- Published by Springer Nature
- p. 869-902
INDICES: Applying DDDAS Principles for Performance Interference-aware Cloud-to-Fog Application Migration
- 31 March 2023
- Published by Springer Nature
- p. 869-902
INDICES: Applying DDDAS Principles for Performance Interference-aware Cloud-to-Fog Application Migration
- Open Access
DMSConfig: Automated Configuration Tuning for Distributed IoT Message Systems Using Deep Reinforcement Learning
The Distributed Messaging Systems (DMSs) used in IoT systems require timely and reliable data dissemination, which can be achieved through configurable parameters. However, the high-dimensional configuration space makes it difficult for users to find the best options that maximize application throughput while meeting specific latency constraints. Existing approaches to automatic software profiling have limitations, such as only optimizing throughput, not guaranteeing explicit latency limitations, and resulting in local optima due to discretizing parameter ranges. To overcome these challenges, a novel configuration tuning system called DMSConfig is proposed that uses machine learning and deep reinforcement learning. DMSConfig interacts with a data-driven environment prediction model, avoiding the cost of online interactions with the production environment. DMSConfig employs the deep deterministic policy gradient (DDPG) method and a custom reward mechanism to make configuration decisions based on predicted DMS states and performance. Experiments show that DMSConfig performs significantly better than the default configuration, is highly adaptive to serve tuning requests with different latency boundaries, and has similar throughput to prevalent parameter tuning tools with fewer latency violations.
Open AccessDMSConfig: Automated Configuration Tuning for Distributed IoT Message Systems Using Deep Reinforcement Learning
The Distributed Messaging Systems (DMSs) used in IoT systems require timely and reliable data dissemination, which can be achieved through configurable parameters. However, the high-dimensional configuration space makes it difficult for users to find the best options that maximize application throughput while meeting specific latency constraints. Existing approaches to automatic software profiling have limitations, such as only optimizing throughput, not guaranteeing explicit latency limitations, and resulting in local optima due to discretizing parameter ranges. To overcome these challenges, a novel configuration tuning system called DMSConfig is proposed that uses machine learning and deep reinforcement learning. DMSConfig interacts with a data-driven environment prediction model, avoiding the cost of online interactions with the production environment. DMSConfig employs the deep deterministic policy gradient (DDPG) method and a custom reward mechanism to make configuration decisions based on predicted DMS states and performance. Experiments show that DMSConfig performs significantly better than the default configuration, is highly adaptive to serve tuning requests with different latency boundaries, and has similar throughput to prevalent parameter tuning tools with fewer latency violations.
- Open Access
Risk-Aware Scene Sampling for Dynamic Assurance of Autonomous Systems
Autonomous Cyber-Physical Systems must often operate under uncertainties like sensor degradation and shifts in the operating conditions, which increases its operational risk. Dynamic Assurance of these systems requires designing runtime safety components like Out-of-Distribution detectors and risk estimators, which require labeled data from different operating modes of the system that belong to scenes with adverse operating conditions, sensors, and actuator faults. Collecting real-world data of these scenes can be expensive and sometimes not feasible. So, scenario description languages with samplers like random and grid search are available to generate synthetic data from simulators, replicating these real-world scenes. However, we point out three limitations in using these conventional samplers. First, they are passive samplers, which do not use the feedback of previous results in the sampling process. Second, the variables to be sampled may have constraints that are often not included. Third, they do not balance the tradeoff between exploration and exploitation, which we hypothesize is necessary for better search space coverage. We present a scene generation approach with two samplers called Random Neighborhood Search (RNS) and Guided Bayesian Optimization (GBO), which extend the conventional random search and Bayesian Optimization search to include the limitations. Also, to facilitate the samplers, we use a risk-based metric that evaluates how risky the scene was for the system. We demonstrate our approach using an Autonomous Vehicle example in CARLA simulation. To evaluate our samplers, we compared them against the baselines of random search, grid search, and Halton sequence search. Our samplers of RNS and GBO sampled a higher percentage of high-risk scenes of 83% and 92%, compared to 56%, 66% and 71% of the grid, random and Halton samplers, respectively.
Open AccessRisk-Aware Scene Sampling for Dynamic Assurance of Autonomous Systems
Autonomous Cyber-Physical Systems must often operate under uncertainties like sensor degradation and shifts in the operating conditions, which increases its operational risk. Dynamic Assurance of these systems requires designing runtime safety components like Out-of-Distribution detectors and risk estimators, which require labeled data from different operating modes of the system that belong to scenes with adverse operating conditions, sensors, and actuator faults. Collecting real-world data of these scenes can be expensive and sometimes not feasible. So, scenario description languages with samplers like random and grid search are available to generate synthetic data from simulators, replicating these real-world scenes. However, we point out three limitations in using these conventional samplers. First, they are passive samplers, which do not use the feedback of previous results in the sampling process. Second, the variables to be sampled may have constraints that are often not included. Third, they do not balance the tradeoff between exploration and exploitation, which we hypothesize is necessary for better search space coverage. We present a scene generation approach with two samplers called Random Neighborhood Search (RNS) and Guided Bayesian Optimization (GBO), which extend the conventional random search and Bayesian Optimization search to include the limitations. Also, to facilitate the samplers, we use a risk-based metric that evaluates how risky the scene was for the system. We demonstrate our approach using an Autonomous Vehicle example in CARLA simulation. To evaluate our samplers, we compared them against the baselines of random search, grid search, and Halton sequence search. Our samplers of RNS and GBO sampled a higher percentage of high-risk scenes of 83% and 92%, compared to 56%, 66% and 71% of the grid, random and Halton samplers, respectively.
- 1 May 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 184-191
EXPPO: EXecution Performance Profiling and Optimization for CPS Co-simulation-as-a-Service
A co-simulation may comprise several heterogeneous federates with diverse spatial and temporal execution characteristics. In an iterative time-stepped simulation, a federation exhibits the Bulk Synchronous Parallel (BSP) computation paradigm in which all federates perform local operations and synchronize with their peers before proceeding to the next round of computation. In this context, the lowest performing (i.e., slowest) federate dictates the progression of the federation logical time. One challenge in co-simulation is performance profiling for individual federates and entire federations. The computational resource assignment to the federates can have a large impact on federation performance. Furthermore, a federation may comprise federates located on different physical machines as is the case for cloud and edge computing environments. As such, distributed profiling and resource assignment to the federation is a major challenge for operationalizing the co-simulation execution at scale. This paper presents the Execution Performance Profiling and Optimization (EXPPO) methodology, which addresses these challenges by using execution performance profiling at each simulation execution step and for every federate in a federation. EXPPO uses profiling to learn performance models for each federate, and uses these models in its federation resource recommendation tool to solve an optimization problem that improves the execution performance of the co-simulation. Using an experimental testbed, the efficacy of EXPPO is validated to show the benefits of performance profiling and resource assignment in improving the execution runtimes of co-simulations while also minimizing the execution cost.
- 1 May 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 184-191
EXPPO: EXecution Performance Profiling and Optimization for CPS Co-simulation-as-a-Service
A co-simulation may comprise several heterogeneous federates with diverse spatial and temporal execution characteristics. In an iterative time-stepped simulation, a federation exhibits the Bulk Synchronous Parallel (BSP) computation paradigm in which all federates perform local operations and synchronize with their peers before proceeding to the next round of computation. In this context, the lowest performing (i.e., slowest) federate dictates the progression of the federation logical time. One challenge in co-simulation is performance profiling for individual federates and entire federations. The computational resource assignment to the federates can have a large impact on federation performance. Furthermore, a federation may comprise federates located on different physical machines as is the case for cloud and edge computing environments. As such, distributed profiling and resource assignment to the federation is a major challenge for operationalizing the co-simulation execution at scale. This paper presents the Execution Performance Profiling and Optimization (EXPPO) methodology, which addresses these challenges by using execution performance profiling at each simulation execution step and for every federate in a federation. EXPPO uses profiling to learn performance models for each federate, and uses these models in its federation resource recommendation tool to solve an optimization problem that improves the execution performance of the co-simulation. Using an experimental testbed, the efficacy of EXPPO is validated to show the benefits of performance profiling and resource assignment in improving the execution runtimes of co-simulations while also minimizing the execution cost.
- 1 December 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1607-1612
STRATUM: A BigData-as-a-Service for Lifecycle Management of IoT Analytics Applications
Smart Internet of Things (IoT) applications require real-time and robust predictive analytics, which are based on Machine Learning (ML) models. Building ML models from Big Data is not only time-consuming, but developers often lack the needed expertise for feature engineering, parameter tuning, and model selection. The proliferation of ML libraries and frameworks, data ingestion tools, stream and batch processing engines, visualization techniques, and the range of available hardware platforms further exacerbates the system design, rapid development, and deployment problems. Finally, resource constraints of IoT require that the execution of the analytics engine be distributed across the cloud-edge spectrum. To overcome these daunting challenges, we present Stratum, which is an event-driven Big Data-as-a-Service offering for IoT analytics lifecycle management. Stratum provides users with an intuitive, declarative mechanism based on the principles of model-driven engineering to specify the application and infrastructure requirements. It automates the deployment via generative programming principles. This paper highlights the problems that Stratum resolves, demonstrating its capabilities using real-world case studies.
- 1 December 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1607-1612
STRATUM: A BigData-as-a-Service for Lifecycle Management of IoT Analytics Applications
Smart Internet of Things (IoT) applications require real-time and robust predictive analytics, which are based on Machine Learning (ML) models. Building ML models from Big Data is not only time-consuming, but developers often lack the needed expertise for feature engineering, parameter tuning, and model selection. The proliferation of ML libraries and frameworks, data ingestion tools, stream and batch processing engines, visualization techniques, and the range of available hardware platforms further exacerbates the system design, rapid development, and deployment problems. Finally, resource constraints of IoT require that the execution of the analytics engine be distributed across the cloud-edge spectrum. To overcome these daunting challenges, we present Stratum, which is an event-driven Big Data-as-a-Service offering for IoT analytics lifecycle management. Stratum provides users with an intuitive, declarative mechanism based on the principles of model-driven engineering to specify the application and infrastructure requirements. It automates the deployment via generative programming principles. This paper highlights the problems that Stratum resolves, demonstrating its capabilities using real-world case studies.
- Open Access
- 7 November 2019
- Published by Association for Computing Machinery (ACM)
- p. 1-14
Linearize, predict and place
Many IoT applications found in cyber-physical systems, such as smart grids, must take control actions in response to critical events, such as supply-demand mismatch, which requires low-latency processing of streaming data for rapid event detection and anomaly remediation. These streaming applications generally take the form of directed acyclic graphs (DAGs), where vertices represent operators and edges represent the flow of data between these operators. Edge computing has recently attracted significant attention as a means to readily meet the requirements of latency-critical IoT applications due to its ability to provide low-latency processing near the source of data. To accrue the benefits of edge computing, the constituent operators of these applications must be placed in a manner that intelligently trades-off inter-operator communication costs with the cost of interference incurred due to co-location of operators on the same resource-constrained edge devices. To address these challenges and to substantially simplify the placement problem for DAGs of arbitrary sizes and topologies, we present an algorithm that first transforms any arbitrary stream processing DAG into an approximate set of linear chains. Subsequently, a data-driven latency prediction model for co-located linear chains is used to inform the placement of operators such that the makespan, defined as the maximum latency of all paths in the DAG, is minimized. We empirically evaluate our algorithm using a variety of DAG placement scenarios on a Beagle Bone cluster, which is representative of an edge computing environment.
- 7 November 2019
- Published by Association for Computing Machinery (ACM)
- p. 1-14
Open AccessLinearize, predict and place
Many IoT applications found in cyber-physical systems, such as smart grids, must take control actions in response to critical events, such as supply-demand mismatch, which requires low-latency processing of streaming data for rapid event detection and anomaly remediation. These streaming applications generally take the form of directed acyclic graphs (DAGs), where vertices represent operators and edges represent the flow of data between these operators. Edge computing has recently attracted significant attention as a means to readily meet the requirements of latency-critical IoT applications due to its ability to provide low-latency processing near the source of data. To accrue the benefits of edge computing, the constituent operators of these applications must be placed in a manner that intelligently trades-off inter-operator communication costs with the cost of interference incurred due to co-location of operators on the same resource-constrained edge devices. To address these challenges and to substantially simplify the placement problem for DAGs of arbitrary sizes and topologies, we present an algorithm that first transforms any arbitrary stream processing DAG into an approximate set of linear chains. Subsequently, a data-driven latency prediction model for co-located linear chains is used to inform the placement of operators such that the makespan, defined as the maximum latency of all paths in the DAG, is minimized. We empirically evaluate our algorithm using a variety of DAG placement scenarios on a Beagle Bone cluster, which is representative of an edge computing environment.
- 1 December 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1337-1346
Technology Enablers for Big Data, Multi-Stage Analysis in Medical Image Processing
Big data medical image processing applications involving multi-stage analysis often exhibit significant variability in processing times ranging from a few seconds to several days. Moreover, due to the sequential nature of executing the analysis stages enforced by traditional software technologies and platforms, any errors in the pipeline are only detected at the later stages despite the sources of errors predominantly being the highly compute-intensive first stage. This wastes precious computing resources and incurs prohibitively higher costs for re-executing the application. The medical image processing community to date remains largely unaware of these issues and continues to use traditional high-performance computing clusters, which incur a high operating cost due to the use of dedicated resources and expensive centralized file systems. To overcome these challenges, this paper proposes an alternative approach for multi-stage analysis in medical image processing by using the Apache Hadoop ecosystem and offering it as a service in the cloud. We make the following contributions. First, we propose a concurrent pipeline execution framework and an associated semi-automatic, real-time monitoring and checkpointing framework that can detect outliers and achieve quality assurance without having to completely execute the expensive first stage of processing thereby expediting the entire multi-stage analysis. Second, we present a simulator to rapidly estimate the execution time for a given multi-stage analysis, which can aid the users in deciding the appropriate approach for their use cases. We conduct empirical evaluation of our framework and show that it requires 76.75% lesser wall time and 29.22% lesser resource time compared to the traditional approach that lacks such a quality assurance mechanism.
- 1 December 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1337-1346
Technology Enablers for Big Data, Multi-Stage Analysis in Medical Image Processing
Big data medical image processing applications involving multi-stage analysis often exhibit significant variability in processing times ranging from a few seconds to several days. Moreover, due to the sequential nature of executing the analysis stages enforced by traditional software technologies and platforms, any errors in the pipeline are only detected at the later stages despite the sources of errors predominantly being the highly compute-intensive first stage. This wastes precious computing resources and incurs prohibitively higher costs for re-executing the application. The medical image processing community to date remains largely unaware of these issues and continues to use traditional high-performance computing clusters, which incur a high operating cost due to the use of dedicated resources and expensive centralized file systems. To overcome these challenges, this paper proposes an alternative approach for multi-stage analysis in medical image processing by using the Apache Hadoop ecosystem and offering it as a service in the cloud. We make the following contributions. First, we propose a concurrent pipeline execution framework and an associated semi-automatic, real-time monitoring and checkpointing framework that can detect outliers and achieve quality assurance without having to completely execute the expensive first stage of processing thereby expediting the entire multi-stage analysis. Second, we present a simulator to rapidly estimate the execution time for a given multi-stage analysis, which can aid the users in deciding the appropriate approach for their use cases. We conduct empirical evaluation of our framework and show that it requires 76.75% lesser wall time and 29.22% lesser resource time compared to the traditional approach that lacks such a quality assurance mechanism.
- 1 December 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 109-114
A Model-Driven Approach to Automate the Deployment and Management of Cloud Services
Although many provisioning tools are available for deployment and management of composite cloud services to overcome the manual efforts that are tedious and error-prone, users are often required to specify Infrastructure-as-Code (IAC) solutions via low-level scripting. IAC demands domain knowledge for provisioning the services across heterogeneous cloud platforms and incurs a steep learning curve. To address these challenges, we present a technology-and platform-agnostic self-service framework called CloudCAMP. It incorporates domain-specific modeling so that the specifications and dependencies imposed by the cloud platform and application architecture can be specified at an intuitive, higher level of abstraction without the need for domain expertise. CloudCAMP transforms the partial specifications into deployable Infrastructure-as-Code (IAC) using the Transformational-Generative paradigm and by leveraging an extensible and reusable knowledge base. The auto-generated IAC can be handled by existing tools to provision the services components automatically. We validate our approach quantitatively by showing a comparative study of savings in manual and scripting efforts versus using CloudCAMP.
- 1 December 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 109-114
A Model-Driven Approach to Automate the Deployment and Management of Cloud Services
Although many provisioning tools are available for deployment and management of composite cloud services to overcome the manual efforts that are tedious and error-prone, users are often required to specify Infrastructure-as-Code (IAC) solutions via low-level scripting. IAC demands domain knowledge for provisioning the services across heterogeneous cloud platforms and incurs a steep learning curve. To address these challenges, we present a technology-and platform-agnostic self-service framework called CloudCAMP. It incorporates domain-specific modeling so that the specifications and dependencies imposed by the cloud platform and application architecture can be specified at an intuitive, higher level of abstraction without the need for domain expertise. CloudCAMP transforms the partial specifications into deployable Infrastructure-as-Code (IAC) using the Transformational-Generative paradigm and by leveraging an extensible and reusable knowledge base. The auto-generated IAC can be handled by existing tools to provision the services components automatically. We validate our approach quantitatively by showing a comparative study of savings in manual and scripting efforts versus using CloudCAMP.
- 1 December 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-10
UPSARA: A Model-Driven Approach for Performance Analysis of Cloud-Hosted Applications
Accurately analyzing the sources of performance anomalies in cloud-based applications is a hard problem due both to the multi tenant nature of cloud deployment and changing application workloads. To that end many different resource instrumentation and application performance modeling frameworks have been developed in recent years to help in the effective deployment and resource management decisions. Yet, the significant differences among these frameworks in terms of their APIs, their ability to instrument resources at different levels of granularity, and making sense of the collected information make it extremely hard to effectively use these frameworks. Not addressing these complexities can result in operators providing incompatible and incorrect configurations leading to inaccurate diagnosis of performance issues and hence incorrect resource management. To address these challenges, we present UPSARA, a model-driven generative framework that provides an extensible, lightweight and scalable performance monitoring, analysis and testing framework for cloud-hosted applications. UPSARA helps alleviate the accidental complexities in configuring the right resource monitoring and performance testing strategies for the underlying instrumentation frameworks used. We evaluate the effectiveness of UPSARA in the context of representative use cases highlighting its features and benefits.
- 1 December 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-10
UPSARA: A Model-Driven Approach for Performance Analysis of Cloud-Hosted Applications
Accurately analyzing the sources of performance anomalies in cloud-based applications is a hard problem due both to the multi tenant nature of cloud deployment and changing application workloads. To that end many different resource instrumentation and application performance modeling frameworks have been developed in recent years to help in the effective deployment and resource management decisions. Yet, the significant differences among these frameworks in terms of their APIs, their ability to instrument resources at different levels of granularity, and making sense of the collected information make it extremely hard to effectively use these frameworks. Not addressing these complexities can result in operators providing incompatible and incorrect configurations leading to inaccurate diagnosis of performance issues and hence incorrect resource management. To address these challenges, we present UPSARA, a model-driven generative framework that provides an extensible, lightweight and scalable performance monitoring, analysis and testing framework for cloud-hosted applications. UPSARA helps alleviate the accidental complexities in configuring the right resource monitoring and performance testing strategies for the underlying instrumentation frameworks used. We evaluate the effectiveness of UPSARA in the context of representative use cases highlighting its features and benefits.