Dengyao Mo
Research Subjects
Dengyao Mo has a total of 20 co-authors. Here are the top co-authors by number of shared publications:
- Samuel. H. Huang
University of Cincinnati
3 shared publications - Aoxiao Zhong
Havard Univerrsity, Cambridge, USA
1 shared publication - Dian Ding
Shanghai Jiao Tong University, Shanghai, China
1 shared publication - Erci Xu
Shanghai Jiao Tong University, Shanghai, China
1 shared publication - Guangtao Xue
Shanghai Jiao Tong University, Shanghai, China
1 shared publication - Guiyang Liu
Alibaba Group, Hangzhou, China
1 shared publication
Latest Publications
- Open Access
- 20 March 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Architecture and Code Optimization
- Vol. 22 (1) , 1-25
MasterPlan: A Reinforcement Learning Based Scheduler for Archive Storage
With the sheer volume of data in today’s world, archive storage systems play a significant role in persisting the cold data. Due to stringent cost concerns, one popular design is to organize disks into groups and periodically switch them to be powered on for serving user requests. Scheduling thus becomes critical for both CapEx and performance. Unfortunately, field results indicate that existing schedulers can be often suboptimal. Our further analysis suggests that the main reason is the mismatch between the ever-changing workloads and the fixed set of coarsely-configured parameters in current heuristic-based schedulers. In this article, we propose MasterPlan , a reinforcement learning (RL) based scheduler for archive storage systems. By identifying the unique characteristics of archive storage service, we design a state space and reward function for the RL agent. MasterPlan includes a continuous action encoding approach to guarantee efficient exploration, and a meta adaptation module to extract features of workload series. Experiments show that MasterPlan can achieve 1.25× throughput, 2.16× 99 th latency and 1.47× power draw improvement compared to existing solutions.
- 20 March 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Architecture and Code Optimization
- Vol. 22 (1) , 1-25
Open AccessMasterPlan: A Reinforcement Learning Based Scheduler for Archive Storage
With the sheer volume of data in today’s world, archive storage systems play a significant role in persisting the cold data. Due to stringent cost concerns, one popular design is to organize disks into groups and periodically switch them to be powered on for serving user requests. Scheduling thus becomes critical for both CapEx and performance. Unfortunately, field results indicate that existing schedulers can be often suboptimal. Our further analysis suggests that the main reason is the mismatch between the ever-changing workloads and the fixed set of coarsely-configured parameters in current heuristic-based schedulers. In this article, we propose MasterPlan , a reinforcement learning (RL) based scheduler for archive storage systems. By identifying the unique characteristics of archive storage service, we design a state space and reward function for the RL agent. MasterPlan includes a continuous action encoding approach to guarantee efficient exploration, and a meta adaptation module to extract features of workload series. Experiments show that MasterPlan can achieve 1.25× throughput, 2.16× 99 th latency and 1.47× power draw improvement compared to existing solutions.
- Open Access
- 24 August 2024
- Published by Association for Computing Machinery (ACM)
- p. 4559-4570
LogParser-LLM: Advancing Efficient Log Parsing with Large Language Models
Logs are ubiquitous digital footprints, playing an indispensable role in system diagnostics, security analysis, and performance optimization. The extraction of actionable insights from logs is critically dependent on the log parsing process, which converts raw logs into structured formats for downstream analysis. Yet, the complexities of contemporary systems and the dynamic nature of logs pose significant challenges to existing automatic parsing techniques. The emergence of Large Language Models (LLM) offers new horizons. With their expansive knowledge and contextual prowess, LLMs have been transformative across diverse applications. Building on this, we introduce LogParser-LLM, a novel log parser integrated with LLM capabilities. This union seamlessly blends semantic insights with statistical nuances, obviating the need for hyper-parameter tuning and labeled training data, while ensuring rapid adaptability through online parsing. Further deepening our exploration, we address the intricate challenge of parsing granularity, proposing a new metric and integrating human interactions to allow users to calibrate granularity to their specific needs. Our method's efficacy is empirically demonstrated through evaluations on the Loghub-2k and the large-scale LogPub benchmark. In evaluations on the LogPub benchmark, involving an average of 3.6 million logs per dataset across 14 datasets, our LogParser-LLM requires only 272.5 LLM invocations on average, achieving a 90.6% F1 score for grouping accuracy and an 81.1% for parsing accuracy. These results demonstrate the method's high efficiency and accuracy, outperforming current state-of-the-art log parsers, including pattern-based, neural network-based, and existing LLM-enhanced approaches.
- 24 August 2024
- Published by Association for Computing Machinery (ACM)
- p. 4559-4570
Open AccessLogParser-LLM: Advancing Efficient Log Parsing with Large Language Models
Logs are ubiquitous digital footprints, playing an indispensable role in system diagnostics, security analysis, and performance optimization. The extraction of actionable insights from logs is critically dependent on the log parsing process, which converts raw logs into structured formats for downstream analysis. Yet, the complexities of contemporary systems and the dynamic nature of logs pose significant challenges to existing automatic parsing techniques. The emergence of Large Language Models (LLM) offers new horizons. With their expansive knowledge and contextual prowess, LLMs have been transformative across diverse applications. Building on this, we introduce LogParser-LLM, a novel log parser integrated with LLM capabilities. This union seamlessly blends semantic insights with statistical nuances, obviating the need for hyper-parameter tuning and labeled training data, while ensuring rapid adaptability through online parsing. Further deepening our exploration, we address the intricate challenge of parsing granularity, proposing a new metric and integrating human interactions to allow users to calibrate granularity to their specific needs. Our method's efficacy is empirically demonstrated through evaluations on the Loghub-2k and the large-scale LogPub benchmark. In evaluations on the LogPub benchmark, involving an average of 3.6 million logs per dataset across 14 datasets, our LogParser-LLM requires only 272.5 LLM invocations on average, achieving a 90.6% F1 score for grouping accuracy and an 81.1% for parsing accuracy. These results demonstrate the method's high efficiency and accuracy, outperforming current state-of-the-art log parsers, including pattern-based, neural network-based, and existing LLM-enhanced approaches.
- 1 May 2012
- Published by Elsevier in Artificial Intelligence in Medicine
- Vol. 55 (1) , 51-59
Identifying a small set of marker genes using minimum expected cost of misclassification
- 1 May 2012
- Published by Elsevier in Artificial Intelligence in Medicine
- Vol. 55 (1) , 51-59
Identifying a small set of marker genes using minimum expected cost of misclassification
- 4 May 2011
- Published by SAGE Publications in Intelligent Data Analysis: An International Journal
- Vol. 15 (3) , 375-398
Feature selection based on inference correlation
Feature selection is a critical preprocessing step in machine learning. It contributes to cost-effective model building and improvement of model prediction performance. Generally, a feature selection algorithm requires a dependency measure and a sear
- 4 May 2011
- Published by SAGE Publications in Intelligent Data Analysis: An International Journal
- Vol. 15 (3) , 375-398
Feature selection based on inference correlation
Feature selection is a critical preprocessing step in machine learning. It contributes to cost-effective model building and improvement of model prediction performance. Generally, a feature selection algorithm requires a dependency measure and a sear
- 18 February 2011
- Published by Wiley in WIREs Data Mining and Knowledge Discovery
- Vol. 1 (2) , 154-163
Mining large medical claims database to identify high‐risk patients: The case of antidepressant utilization
- 18 February 2011
- Published by Wiley in WIREs Data Mining and Knowledge Discovery
- Vol. 1 (2) , 154-163
Mining large medical claims database to identify high‐risk patients: The case of antidepressant utilization
- 9 November 2010
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Knowledge and Data Engineering
- Vol. 24 (1) , 59-71
Fractal-Based Intrinsic Dimension Estimation and Its Application in Dimensionality Reduction
Dimensionality reduction is an important step in knowledge discovery in databases. Intrinsic dimension indicates the number of variables necessary to describe a data set. Two methods, box-counting dimension and correlation dimension, are commonly used for intrinsic dimension estimation. However, the robustness of these two methods has not been rigorously studied. This paper demonstrates that correlation dimension is more robust with respect to data sample size. In addition, instead of using a user selected distance d, we propose a new approach to capture all log-log pairs of a data set to more precisely estimate the correlation dimension. Systematic experiments are conducted to study factors that influence the computation of correlation dimension, including sample size, the number of redundant variables, and the portion of log-log plot used for calculation. Experiments on real-world data sets confirm the effectiveness of intrinsic dimension estimation with our improved method. Furthermore, a new supervised dimensionality reduction method based on intrinsic dimension estimation was introduced and validated.
- 9 November 2010
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Knowledge and Data Engineering
- Vol. 24 (1) , 59-71
Fractal-Based Intrinsic Dimension Estimation and Its Application in Dimensionality Reduction
Dimensionality reduction is an important step in knowledge discovery in databases. Intrinsic dimension indicates the number of variables necessary to describe a data set. Two methods, box-counting dimension and correlation dimension, are commonly used for intrinsic dimension estimation. However, the robustness of these two methods has not been rigorously studied. This paper demonstrates that correlation dimension is more robust with respect to data sample size. In addition, instead of using a user selected distance d, we propose a new approach to capture all log-log pairs of a data set to more precisely estimate the correlation dimension. Systematic experiments are conducted to study factors that influence the computation of correlation dimension, including sample size, the number of redundant variables, and the portion of log-log plot used for calculation. Experiments on real-world data sets confirm the effectiveness of intrinsic dimension estimation with our improved method. Furthermore, a new supervised dimensionality reduction method based on intrinsic dimension estimation was introduced and validated.