ACAlberto Cerpa

Alberto Cerpa

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Alberto Cerpa has a total of 45 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Scalable and Efficient Reinforcement Learning for Virtual Machine Rescheduling in Cloud Data Centers

    by Xianzhong Ding, Yunkai Zhang, Binbin Chen, Donghao Ying, Tieying Zhang, Jianjun Chen, Lei Zhang, Alberto Cerpa, Wan Du

    Managing a vast number of virtual machines (VMs) efficiently is a critical challenge in modern large-scale data centers. The continuous creation and termination of VMs lead to resource fragmentation across physical machines (PMs), necessitating periodic VM rescheduling to optimize resource utilization. Despite its significance, VM rescheduling has received limited attention in the literature. A key challenge is that, unlike conventional combinatorial optimization problems, the efficiency of rescheduling algorithms is heavily impacted by inference time, as VM states evolve dynamically during execution. This scalability bottleneck hampers existing methods. To address this, we propose VMR$^{2}$L, a reinforcement learning framework tailored for VM rescheduling. VMR$^{2}$L integrates a two-stage decision-making process to accommodate complex operational constraints, a feature extraction mechanism that captures critical relational information for rescheduling, and a risk-aware evaluation strategy that enables users to balance execution speed and rescheduling accuracy. Extensive experiments using real-world data from a production-scale data center demonstrate that VMR$^{2}$L achieves near-optimal performance while reducing inference time to a matter of seconds.


  • EDRP: Enhanced Dynamic Relay Point Protocol for Data Dissemination in Multihop Wireless IoT Networks

    by Jothi Prasanna Shanmuga Sundaram, Magzhan Gabidolla, Luis Fujarte, Shawn Duong, Jianlin Guo, Toshiaki Koike-Akino, Pu Wang, Kieran Parsons, Philip V. Orlik, Takenori Sumi, et al.

    Emerging Internet of Things (IoT) applications are transitioning from battery-powered to grid-powered nodes. DRP, a contention-based data dissemination protocol, was developed for these applications. Traditional contention-based protocols resolve collisions through control packet exchanges, significantly reducing goodput. DRP mitigates this issue by employing a distributed delay timer mechanism that assigns transmission-start delays based on the average link quality (LQ) between a sender and its children, prioritizing highly connected nodes for early transmission. However, our in-field experiments reveal that DRP is unable to accommodate real-world LQ fluctuations, leading to overlapping transmissions from multiple senders. This overlap triggers CSMA’s random back-off delays, ultimately degrading the goodput performance. To address these shortcomings, we first conduct a theoretical analysis that characterizes the design requirements induced by real-world LQ fluctuations and DRP’s passive acknowledgments. Guided by this analysis, we design enhanced-DRP (EDRP), which integrates two novel components: 1) LQ-aware CSMA (LQ-CSMA) and 2) a machine learning-based block size selection (ML-BSS) algorithm for rateless codes. LQ-CSMA dynamically restricts the back-off delay range based on real-time LQ estimates, ensuring that nodes with stronger connectivity experience shorter delays. ML-BSS algorithm predicts future LQ conditions and optimally adjusts the block size for rateless coding, reducing overhead and enhancing goodput. In-field evaluations of EDRP demonstrate an average goodput improvement of 39.43% over the competing protocols.

  • Open Access

    EDRP: Enhanced Dynamic Relay Point Protocol for Data Dissemination in Multi-hop Wireless IoT Networks

    by Jothi Prasanna Shanmuga Sundaram, Magzhan Gabidolla, Luis Fujarte, Shawn Duong, Jianlin Guo, Toshiaki Koike-Akino, Pu, Perry Wang, Kieran Parsons, Philip V. Orlik, et al.

    Emerging IoT applications are transitioning from battery-powered to grid-powered nodes. DRP, a contention-based data dissemination protocol, was developed for these applications. Traditional contention-based protocols resolve collisions through control packet exchanges, significantly reducing goodput. DRP mitigates this issue by employing a distributed delay timer mechanism that assigns transmission-start delays based on the average link quality between a sender and its children, prioritizing highly connected nodes for early transmission. However, our in-field experiments reveal that DRP is unable to accommodate real-world link quality fluctuations, leading to overlapping transmissions from multiple senders. This overlap triggers CSMA's random back-off delays, ultimately degrading the goodput performance. To address these shortcomings, we first conduct a theoretical analysis that characterizes the design requirements induced by real-world link quality fluctuations and DRP's passive acknowledgments. Guided by this analysis, we design EDRP, which integrates two novel components: (i) Link-Quality Aware CSMA (LQ-CSMA) and (ii) a Machine Learning-based Block Size Selection (ML-BSS) algorithm for rateless codes. LQ-CSMA dynamically restricts the back-off delay range based on real-time link quality estimates, ensuring that nodes with stronger connectivity experience shorter delays. ML-BSS algorithm predicts future link quality conditions and optimally adjusts the block size for rateless coding, reducing overhead and enhancing goodput. In-field evaluations of EDRP demonstrate an average goodput improvement of 39.43\% than the competing protocols.

  • Open Access

    Towards VM Rescheduling Optimization Through Deep Reinforcement Learning

    by Xianzhong Ding, Yunkai Zhang, Binbin Chen, Donghao Ying, Tieying Zhang, Jianjun Chen, Lei Zhang, Alberto Cerpa, Wan Du


  • Open Access

    COMNETS: COst-sensitive decision trees approach to throughput optimization for Multi-radio IoT NETworkS

    by Jothi Prasanna Shanmuga Sundaram, Magzhan Gabidolla, Miguel A. Carreira-Perpinan, Alberto E. Cerpa

    Mesoscale IoT applications, such as P2P energy trade and real-time industrial control systems, demand high throughput and low latency, with a secondary emphasis on energy efficiency as they rely on grid power or large-capacity batteries. MARS, a multi-radio architecture, leverages ML to instantaneously select the optimal radio for transmission, outperforming the single-radio systems. However, MARS encounters a significant issue with cost sensitivity, where high-cost errors account for 40% throughput loss. Current cost-sensitive ML algorithms assign a misclassification cost for each class but not for each data sample. In MARS, each data sample has different costs, making it tedious to employ existing cost-sensitive ML algorithms. First, we address this issue by developing COMNETS, an ML-based radio selector using oblique trees optimized by Tree Alternating Optimization (TAO). TAO incorporates sample-specific misclassification costs to avert high-cost errors and achieves a 50% reduction in the decision tree size, making it more suitable for resource-constrained IoT devices. Second, we prove the stability property of TAO and leverage it to understand the critical factors affecting the radio-selection problem. Finally, our real-world evaluation of COMNETS at two different locations shows an average throughput gain of 20.83%, 17.39% than MARS.

  • Open Access

    MARS: Multi-radio Architecture with Radio Selection using Decision Trees for emerging mesoscale CPS/IoT applications

    by Jothi Prasanna Shanmuga Sundaram, Arman Zharmagambetov, Magzhan Gabidolla, Miguel A. Carreira-Perpinan, Alberto Cerpa

    IoT is rapidly growing from small-scale apps to large-scale apps. Small-scale apps employ short-range radios like Zigbee,BLE while large-scale apps employ long-range radios like LoRa,NB-IoT. The other upcoming category of apps like P2P energy-trade in smart homes are termed mesoscale IoT apps. There are no specialized radios for these apps. They either use short/long-range radios. To close this gap, we explored mesoscale apps using the COTS IoT radios available. Our qualitative analysis identifies Zigbee and LoRa as potential candidates. Our quantitative analysis on single and multi-hop topologies showed that Zigbee and LoRa achieve competitive throughput at a distance of 500-1200m from the gateway. A fundamental finding of these analyses is that a multi-radio system that can efficiently switch between Zigbee and LoRa performs better than the single-radio systems. However, instantaneously selecting and switching to a high-throughput radio during transmission is not trivial because of the erratic link quality dynamics. To address this issue, we developed MARS, that uses path quality metrics to instantaneously select the high-throughput radio during transmission. However, realizing MARS on resource-constrained end devices entails the challenge of obtaining instantaneous path-quality metrics. Traditional path quality estimation is not instantaneous due to propagation and queuing delays. We overcome this challenge by showing that collecting local path metrics as input to our decision trees provides sufficient information to instantaneously identify the high-throughput radio. The radio selector of MARS is powered by TAO-CART trees. The evaluation of MARS on a large-scale mesh topology at two different locations shows that MARS can efficiently identify and switch to the high-throughput radio during transmission, leading to an average throughput gain of 48.2% and 49.79% over its competitors.

  • Multi-Zone HVAC Control With Model-Based Deep Reinforcement Learning

    by Xianzhong Ding, Alberto Cerpa, Wan Du

    The application of reinforcement learning in controlling Heating, Ventilation, and Air Conditioning (HVAC) systems has been extensively researched. Existing studies primarily focus on Model-Free Reinforcement Learning (MFRL), which involves trial-and-error interactions with real buildings to train the agent. However, MFRL encounters a significant challenge: it requires a large amount of training data to achieve satisfactory performance. While simulation models have been used to generate training data and expedite the training process, they necessitate high-fidelity building models that are difficult to calibrate. As a result, Model-Based Reinforcement Learning (MBRL) has been employed for HVAC control. Although MBRL demonstrates remarkable sample efficiency, it often falls short in terms of asymptotic control performance, particularly in achieving substantial energy savings while ensuring occupants’ thermal comfort. In this study, we conduct experiments to analyze the limitations of current MBRL-based HVAC control methods, focusing on model uncertainty and controller effectiveness. Leveraging the insights gained from these experiments, we develop, an innovative MBRL-based HVAC control system that combines high control performance with exceptional sample efficiency. learns the dynamics of the building by employing an ensemble of environment-conditioned neural networks and utilizes a novel control method called Model Predictive Path Integral (MPPI) for HVAC control. MPPI generates candidate action sequences using an importance sampling weighted algorithm, which is well-suited for multi-zone buildings with high state and action dimensions. We evaluate using EnergyPlus simulations in a five-zone office building, and the results demonstrate that achieves 8.23% higher energy savings compared to the state-of-the-art MBRL solution while maintaining comparable thermal comfort. Moreover, significantly reduces the required training data set by an order of magnitude (10.52 $\times$ ) while delivering performance on par with MFRL approaches. Note to Practitioners —Our research addresses a critical challenge in HVAC control, offering an innovative solution to enhance the data efficiency of HVAC systems while optimizing energy usage. Traditional approaches, such as Model-Free Reinforcement Learning, often require a large volume of real-world data. Our primary focus is improving the effectiveness of HVAC control, a vital aspect of building management that directly affects energy consumption and occupant well-being. We introduce, a Model-Based Reinforcement Learning system designed to significantly improve energy savings while maintaining thermal comfort. achieves remarkable results, offering exceptional sample efficiency and substantially reducing the required training data. Our research leverages an ensemble of environment-conditioned neural networks and employs Model Predictive Path Integral in HVAC control. While presents notable benefits, it also has limitations. Further research and development are required to optimize its performance across different building environments and specific use cases. Future directions should focus on addressing the safety challenges associated with real-world deployment. Beyond HVAC control, the principles and methods explored in this research have potential applications in various automation domains, such as robotics, industrial automation, and manufacturing processes.


  • Open Access

    Exploring Deep Reinforcement Learning for Holistic Smart Building Control

    by Xianzhong Ding, Alberto Cerpa, Wan Du

    In recent years, the focus has been on enhancing user comfort in commercial buildings while cutting energy costs. Efforts have mainly centered on improving HVAC systems, the central control system. However, it’s evident that HVAC alone can’t ensure occupant comfort. Lighting, blinds, and windows, often overlooked, also impact energy use and comfort. This paper introduces a holistic approach to managing the delicate balance between energy efficiency and occupant comfort in commercial buildings. We present OCTOPUS , a system employing a deep reinforcement learning (DRL) framework using data-driven techniques to optimize control sequences for all building subsystems, including HVAC, lighting, blinds, and windows. OCTOPUS ’s DRL architecture features a unique reward function facilitating the exploration of tradeoffs between energy usage and user comfort, effectively addressing the high-dimensional control problem resulting from interactions among these four building subsystems. To meet data training requirements, we emphasize the importance of calibrated simulations that closely replicate target-building operational conditions. We train OCTOPUS using 10-year weather data and a calibrated building model in the EnergyPlus simulator. Extensive simulations demonstrate that OCTOPUS achieves substantial energy savings, outperforming state-of-the-art rule-based and DRL-based methods by 14.26% and 8.1%, respectively, in a LEED Gold Certified building while maintaining desired human comfort levels.


  • Open Access

    TODOS: Thermal sensOr Data-driven Occupancy Estimation System for Smart Buildings

    by Hamid Rajabi, Xianzhong Ding, Wan Du, Alberto Cerpa

    Occupancy sensing and estimation in large commercial buildings has become a significant problem to be solved, with applications ranging from occupancy-based HVAC control to space planning, and security, etc. Thermal sensing is a promising technology to solve this problem, being easy to deploy in practice and allowing an actual occupancy count in a particular room without violating the data and privacy concerns. While initial strides have been made to solve this problem with thermal arrays, there are many problems that remain unsolved, including accuracy performance, overlapping of sensing areas that lead to under/over-counting, and data training requirements for different zones. In this paper, we introduce TODOS 1, a novel system for estimating occupancy in intelligent buildings. TODOS uses a low-cost, low-power thermal sensor array along with a passive infrared sensor. We introduce a novel data processing pipeline that allows us to automatically extract features from the thermal images using an artificial neural network. Through an extensive experimental evaluation2, we show that TODOS provides occupancy detection accuracy of 98% to 100% under different scenarios. In addition, it solves the issue of occupancy over/under-counting by overlapping sensing areas when using multiple thermal sensors in large rooms. This is done by treating the entire area as a single input thermal image instead of partitioning the area into multiple thermal images individually processed. Furthermore, TODOS introduces a data augmentation technique that allows the generation of training data for rooms of different sizes and shapes, without requiring specific training data from each room. Using these data, TODOS can train specifically designed neural networks optimized for any room size and shape, and achieve almost the same level of occupancy detection accuracy in rooms where experimental labeled training data is available, making it a viable solution that generalizes to the different rooms in large buildings.