MYMiao Yu

Miao Yu

*This is an algorithmically generated author record based on publicly available information.

Miao Yu has a total of 38 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Detecting Tiny Performance Regressions at Hyperscale

    by Yang Wang, Mike Chow, Dong Young Yoon, Rohan Bopardikar, Sherry Chen, Zoltan Farkas, Caner Gocmen, Kapil Goenka, Ayichew Hailu, Daniel Hodges, et al.

    This paper presents Meta’s performance testing and monitoring systems, which advance the state of the art in performance regression detection by catching regressions as small as 0.005%. These tiny regressions matter due to our large fleet size and their potential to accumulate over time. Detecting such tiny regressions, however, is challenging, due to various kinds of noises caused by heterogeneous machines, server failures, maintenance operations, load spikes, etc. To detect tiny regressions despite such noises, Meta has developed two complementary systems, ServiceLab and FBDetect. ServiceLab is a pre-production testing platform, which conducts A/B experiments in an isolated environment by reserving servers from our private cloud. While this approach avoids most of the noise in production, heterogeneous machines are still a major challenge, since even machines of the same instance type from our private cloud may have subtle differences in various ways. To address this challenge, we conduct a large-scale study with millions of performance experiments to identify machine factors, such as the kernel, CPU, and datacenter location, that introduce variance to test results. Moreover, we present statistical analysis methods to robustly identify small regressions. Despite the success of ServiceLab, we observe that pre-production testing has its inherent limitation: Due to the limited resource that can be used for testing purpose, pre-production testing cannot fully reproduce the scale and complexity of the production system. As a result, some buggy code or config changes slip through. To capture such buggy changes, FBDetect monitors performance in production as the last line of defense. Unlike ServiceLab, FBDetect cannot control the machines or run repeated experiments. Instead, to battle against noise, FBDetect introduces advanced techniques to capture stack traces fleet-wide, measure fine-grained subroutine-level performance differences, filter out deceptive false-positive regressions, deduplicate correlated regressions, and analyze root causes. Both systems have been in production for over seven years. They detect regressions in thousands of services and ML models running on millions of servers. Each year they detect performance regressions that could otherwise lead to the wastage of millions of machines in the following years. This paper integrates two works, one titled “ServiceLab: Preventing Tiny Performance Regressions at Hyperscale through Pre-Production Testing”, originally published in OSDI 2024, and the other titled “FBDetect: Catching Tiny Performance Regression at Hyperscale through In-Production Monitoring”, originally published in SOSP 2024, both of which were invited for extended publication in ACM TOCS upon recommendation by the Chairs of OSDI 2024 and SOSP 2024, respectively.

  • Open Access

    FBDetect: Catching Tiny Performance Regressions at Hyperscale through In-Production Monitoring

    by Dong Young Yoon, Yang Wang, Miao Yu, Elvis Huang, Juan Ignacio Jones, Abhinay Kukkadapu, Osman Kocas, Jonathan Wiepert, Kapil Goenka, Sherry Chen, et al.

    This paper presents Meta's FBDetect system, which advances the state of the art in performance regression detection by catching regressions as small as 0.005% in noisy production environments. FBDetect monitors around 800,000 time series covering various types of metrics (e.g., throughput, latency, CPU and memory usage) to detect regressions caused by code or configuration changes in hundreds of services running on millions of servers. FBDetect introduces advanced techniques to capture stack traces fleet-wide, measure fine-grained subroutine-level performance differences, filter out deceptive false-positive regressions, deduplicate correlated regressions, and analyze root causes. Beyond these individual techniques, a key strength of FBDetect over prior work is its battle-tested robustness, proven by seven years of production use, and each year catching regressions that would have wasted millions of servers if left undetected.


  • Open Access

    On the Feasibility and Benefits of Extensive Evaluation

    by Yujie Hui, Miao Yu, Hao Qi, Yifan Gan, Tianxi Li, Yuke Li, Xueyuan Ren, Sixiang Ma, Xiaoyi Lu, Yang Wang

    Benchmark and system parameters often have a significant impact on performance evaluation, which raises a long-lasting question about which settings we should use. This paper studies the feasibility and benefits of extensive evaluation. A full extensive evaluation, which tests all possible settings, is usually too expensive. This work investigates whether it is possible to sample a subset of the settings and, upon them, generate observations that match those from a full extensive evaluation. Towards this goal, we have explored the incremental sampling approach, which starts by measuring a small subset of random settings, builds a prediction model on these samples using the popular ANOVA approach, adds more samples if the model is not accurate enough, and terminates otherwise. To summarize our findings: 1) Enhancing a research prototype to support extensive evaluation mostly involves changing hard-coded configurations, which does not take much effort. 2) Some systems are highly predictable, which means that they can achieve accurate predictions with a low sampling rate, but some systems are less predictable. 3) We have not found a method that can consistently outperform random sampling + ANOVA. Based on these findings, we provide recommendations to improve artifact predictability and strategies for selecting parameter values during evaluation.