Zhendong Bei
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Zhendong Bei has a total of 11 co-authors. Here are the top co-authors by number of shared publications:
- Cheng-Zhong Xu
State Key Laboratory of IoTSC, University of Macau, Macau, Macao
3 shared publications - Wen Xiong
Guangzhou Power Supply Bureau, Guangdong Power Grid Co., Ltd., Guangzhou, China
2 shared publications Zhibin Yu
Chinese Academy of Sciences, Shenzhen Institutes of Advanced Technology, Shenzhen, China
2 shared publications- Zhibin Yu
Shenzhen Institutes of Advanced Technology(SIAT), Chinese Academy of Science(CAS), Shenzhen, China
2 shared publications Avi Mendelson
1 shared publicationKai Hwang
Chinese University of Hong Kong (Shenzhen), Hong Kong
1 shared publication
Latest Publications
- 3 March 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computers
- Vol. 71 (4) , 809-823
OSC: An Online Self-Configuring Big Data Framework for Optimization of QoS
Big-data frameworks such as MapReduce/Hadoop or Spark have many performance-critical configuration parameters which may interact with each other in a complex way. Their optimal values for an application on a given cluster are affected by not only the application itself but also its input data. This makes offline auto-configuration approaches hard to be used in practice because the input data of an application may change at each run. To address this issue, we propose an Online Self-Configuring (OSC) approach that automatically determines the optimal parameter values for a given application. OSC synergistically integrates three key techniques. First, OSC leverages ensemble learning to build a precise performance model for a given application. Second, it quantifies the importance of the parameters and interaction intensity between them to accelerate the genetic algorithm for searching optimal configuration parameters. Third, OSC supports an incremental modeling approach to achieve low overhead of the models for online needs. These techniques allow OSC to effectively learn the characteristics of an application and optimize its performance by automatically adjusting the configurations at runtime. Our implementation of OSC atop MapReduce/Hadoop 2.6 improves performance by 60 percent on average and up to 120 percent compared with the state-of-the-art approach. Lastly, the performance benefit of an application running on OSC generally increases along with its input data size.
- 3 March 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computers
- Vol. 71 (4) , 809-823
OSC: An Online Self-Configuring Big Data Framework for Optimization of QoS
Big-data frameworks such as MapReduce/Hadoop or Spark have many performance-critical configuration parameters which may interact with each other in a complex way. Their optimal values for an application on a given cluster are affected by not only the application itself but also its input data. This makes offline auto-configuration approaches hard to be used in practice because the input data of an application may change at each run. To address this issue, we propose an Online Self-Configuring (OSC) approach that automatically determines the optimal parameter values for a given application. OSC synergistically integrates three key techniques. First, OSC leverages ensemble learning to build a precise performance model for a given application. Second, it quantifies the importance of the parameters and interaction intensity between them to accelerate the genetic algorithm for searching optimal configuration parameters. Third, OSC supports an incremental modeling approach to achieve low overhead of the models for online needs. These techniques allow OSC to effectively learn the characteristics of an application and optimize its performance by automatically adjusting the configurations at runtime. Our implementation of OSC atop MapReduce/Hadoop 2.6 improves performance by 60 percent on average and up to 120 percent compared with the state-of-the-art approach. Lastly, the performance benefit of an application running on OSC generally increases along with its input data size.
- 4 October 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Parallel and Distributed Systems
- Vol. 29 (6) , 1371-1384
MIA: Metric Importance Analysis for Big Data Workload Characterization
Data analytics is at the foundation of both high-quality products and services in modern economies and societies. Big data workloads run on complex large-scale computing clusters, which implies significant challenges for deeply understanding and characterizing overall system performance. In general, performance is affected by many factors at multiple layers in the system stack, hence it is challenging to identify the key metrics when characterizing and understanding big data workload performance. In this paper, we propose a novel workload characterization methodology using ensemble learning, called Metric Importance Analysis (MIA), to quantify the respective importance of workload metrics. By focusing on the most important metrics, MIA reduces the complexity of the analysis without losing information. Moreover, we develop the MIA-based Kiviat Plot (MKP) and Benchmark Similarity Matrix (BSM) which provide more insightful information than the traditional linkage clustering based dendrogram to visualize program behavior (dis)similarity. To demonstrate the applicability of MIA, we use it to characterize three big data benchmark suites: HiBench, CloudRank-D and SZTS. The results show that MIA is able to characterize complex big data workloads in a simple, intuitive manner, and reveal interesting insights. Moreover, through a case study, we demonstrate that tuning the configuration parameters related to the important metrics found by MIA results in higher performance improvements than through tuning the parameters related to the less important ones.
- 4 October 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Parallel and Distributed Systems
- Vol. 29 (6) , 1371-1384
MIA: Metric Importance Analysis for Big Data Workload Characterization
Data analytics is at the foundation of both high-quality products and services in modern economies and societies. Big data workloads run on complex large-scale computing clusters, which implies significant challenges for deeply understanding and characterizing overall system performance. In general, performance is affected by many factors at multiple layers in the system stack, hence it is challenging to identify the key metrics when characterizing and understanding big data workload performance. In this paper, we propose a novel workload characterization methodology using ensemble learning, called Metric Importance Analysis (MIA), to quantify the respective importance of workload metrics. By focusing on the most important metrics, MIA reduces the complexity of the analysis without losing information. Moreover, we develop the MIA-based Kiviat Plot (MKP) and Benchmark Similarity Matrix (BSM) which provide more insightful information than the traditional linkage clustering based dendrogram to visualize program behavior (dis)similarity. To demonstrate the applicability of MIA, we use it to characterize three big data benchmark suites: HiBench, CloudRank-D and SZTS. The results show that MIA is able to characterize complex big data workloads in a simple, intuitive manner, and reveal interesting insights. Moreover, through a case study, we demonstrate that tuning the configuration parameters related to the important metrics found by MIA results in higher performance improvements than through tuning the parameters related to the less important ones.
- Open Access
- 16 June 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 5, 13157-13170
ATH: Auto-Tuning HBase’s Configuration via Ensemble Learning
HBase is a distributed database management system and is becoming increasingly popular for applications that need fast random access to a large amount of data. However, it has a number of performancecritical configuration parameters, which may interact with each other in a complex way, making manually tuning them for optimal performance extremely difficult. In this paper, we propose a novel approach to auto-tune the configuration parameters for a given HBase application, called Auto-Tuning HBase (ATH). The key is an accurate performance model with low cost, which takes configuration parameters as inputs. To this end, we systematically explore different modeling techniques and decide to employ an ensemble learning algorithm to build the performance model. Subsequently, we leverage genetic algorithm to search the optimal configuration parameters for the application by using the performance model. As such, ATH can quickly as well as automatically identify a set of configuration parameter values to make the performance of the application optimal. We validate ATH in a cluster with ten nodes by using five typical applications from Yahoo! Cloud Serving Benchmark. The experimental results show that ATH can improve throughput by 41% on average and up to 97% compared with the default configurations. At the same time, the latency of HBase operations is reduced by 11.3% on average and up to 57%.
- 16 June 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 5, 13157-13170
Open AccessATH: Auto-Tuning HBase’s Configuration via Ensemble Learning
HBase is a distributed database management system and is becoming increasingly popular for applications that need fast random access to a large amount of data. However, it has a number of performancecritical configuration parameters, which may interact with each other in a complex way, making manually tuning them for optimal performance extremely difficult. In this paper, we propose a novel approach to auto-tune the configuration parameters for a given HBase application, called Auto-Tuning HBase (ATH). The key is an accurate performance model with low cost, which takes configuration parameters as inputs. To this end, we systematically explore different modeling techniques and decide to employ an ensemble learning algorithm to build the performance model. Subsequently, we leverage genetic algorithm to search the optimal configuration parameters for the application by using the performance model. As such, ATH can quickly as well as automatically identify a set of configuration parameter values to make the performance of the application optimal. We validate ATH in a cluster with ten nodes by using five typical applications from Yahoo! Cloud Serving Benchmark. The experimental results show that ATH can improve throughput by 41% on average and up to 97% compared with the default configurations. At the same time, the latency of HBase operations is reduced by 11.3% on average and up to 57%.
- Open Access
- 23 February 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 5 (99) , 3580-3593
MEST: A Model-Driven Efficient Searching Approach for MapReduce Self-Tuning
Hadoop is the most popular implementation framework of the MapReduce programming model, and it has a number of performance-critical configuration parameters. However, manually setting these parameters to their optimal values not only needs in-depth knowledge on Hadoop as well as the job itself, but also requires a large amount of time and efforts. Automatic approaches have therefore been proposed. Their usage, however, is still quite limited due to the intolerably long searching time. In this paper, we introduce MapreducE Self-Tuning (MEST), a framework that accelerates the searching process for the optimal configuration of a given Hadoop application. We have devised a novel mechanism by integrating the model trees algorithm with the genetic algorithm. As such, MEST significantly reduces the searching time by removing unnecessary profiling, modeling, and searching steps, which are mandatory for existing approaches. Our experiments using five benchmarks, each with two input data sets (DS1 and 2× DS1 ) show that MEST improves the searching efficiency (SE) by factors of 1.37× and 2.18×$ on average respectively over the state-of-the-art approach.
- 23 February 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 5 (99) , 3580-3593
Open AccessMEST: A Model-Driven Efficient Searching Approach for MapReduce Self-Tuning
Hadoop is the most popular implementation framework of the MapReduce programming model, and it has a number of performance-critical configuration parameters. However, manually setting these parameters to their optimal values not only needs in-depth knowledge on Hadoop as well as the job itself, but also requires a large amount of time and efforts. Automatic approaches have therefore been proposed. Their usage, however, is still quite limited due to the intolerably long searching time. In this paper, we introduce MapreducE Self-Tuning (MEST), a framework that accelerates the searching process for the optimal configuration of a given Hadoop application. We have devised a novel mechanism by integrating the model trees algorithm with the genetic algorithm. As such, MEST significantly reduces the searching time by removing unnecessary profiling, modeling, and searching steps, which are mandatory for existing approaches. Our experiments using five benchmarks, each with two input data sets (DS1 and 2× DS1 ) show that MEST improves the searching efficiency (SE) by factors of 1.37× and 2.18×$ on average respectively over the state-of-the-art approach.