PYPing Yang

Ping Yang

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Ping Yang has a total of 34 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    RepoSim: Evaluating Prompt Strategies for Code Completion via User Behavior Simulation

    by Chao Peng, Qinyun Wu, Jiangchao Liu, Jierui Liu, Bo Jiang, Mengqian Xu, Yinghao Wang, Xia Liu, Ping Yang

    Large language models (LLMs) have revolutionized code completion tasks. IDE plugins such as MarsCode can generate code recommendations, saving developers significant time and effort. However, current evaluation methods for code completion are limited by their reliance on static code benchmarks, which do not consider human interactions and evolving repositories. This paper proposes RepoSim, a novel benchmark designed to evaluate code completion tasks by simulating the evolving process of repositories and incorporating user behaviors. RepoSim leverages data from an IDE plugin, by recording and replaying user behaviors to provide a realistic programming context for evaluation. This allows for the assessment of more complex prompt strategies, such as utilizing recently visited files and incorporating user editing history. Additionally, RepoSim proposes a new metric based on users' acceptance or rejection of predictions, offering a user-centric evaluation criterion. Our preliminary evaluation demonstrates that incorporating users' recent edit history into prompts significantly improves the quality of LLM-generated code, highlighting the importance of temporal context in code completion. RepoSim represents a significant advancement in benchmarking tools, offering a realistic and user-focused framework for evaluating code completion performance.


  • Open Access

    CoEdPilot: Recommending Code Edits with Learned Prior Edit Relevance, Project-wise Awareness, and Interactive Nature

    by Chenyan Liu, Yufan Cai, Yun Lin, Yuhuan Huang, Yunrui Pei, Bo Jiang, Ping Yang, Jin Song Dong, Hong Mei

    Recent years have seen the development of LLM-based code generation. Compared to generating code in a software project, incremental code edits are empirically observed to be more frequent. The emerging code editing approaches usually formulate the problem as generating an edit based on known relevant prior edits and context. However, practical code edits can be more complicated. First, an editing session can include multiple (ir)relevant edits to the code under edit. Second, the inference of the subsequent edits is non-trivial as the scope of its ripple effect can be the whole project. In this work, we propose CoEdPilot, an LLM-driven solution to recommend code edits by discriminating the relevant edits, exploring their interactive natures, and estimating its ripple effect in the project. Specifically, CoEdPilot orchestrates multiple neural transformers to identify what and how to edit in the project regarding both edit location and edit content. When a user accomplishes an edit with an optional editing description, an Subsequent Edit Analysis first reports the most relevant files in the project with what types of edits (e.g., keep, insert, and replace) can happen for each line of their code. Next, an Edit-content Generator generates concrete edit options for the lines of code, regarding its relevant prior changes reported by an Edit-dependency Analyzer. Last, both the Subsequent Edit Analysis and the Edit-content Generator capture relevant prior edits as feedback to readjust their recommendations. We train our models by collecting over 180K commits from 471 open-source projects in 5 programming languages. Our extensive experiments show that (1) CoEdPilot can well predict the edits (i.e., predicting edit location with accuracy of 70.8%-85.3%, and the edit content with exact match rate of 41.8% and BLEU4 score of 60.7); (2) CoEdPilot can well boost existing edit generators such as GRACE and CCT5 on exact match rate by 8.57% points and BLEU4 score by 18.08. Last, our user study on 18 participants with 3 editing tasks (1) shows that CoEdPilot can be effective in assisting users to edit code in comparison with Copilot, and (2) sheds light on the future improvement of the tool design. The video demonstration of our tool is available at https://sites.google.com/view/coedpilot/home.


  • Open Access

    Neat: Mobile App Layout Similarity Comparison Based on Graph Convolutional Networks

    by Zhu Tao, Yongqiang Gao, Jiayi Qi, Chao Peng, Qinyun Wu, Xiang Chen, Ping Yang

    A wide variety of device models, screen resolutions and operating systems have emerged with recent advances in mobile devices. As a result, the graphical user interface (GUI) layout in mobile apps has become increasingly complex due to this market fragmentation, with rapid iterations being the norm. Testing page layout issues under these circumstances hence becomes a resource-intensive task, requiring significant manpower and effort due to the vast number of device models and screen resolution adaptations. One of the most challenging issues to cover manually is multi-model and cross-version layout verification for the same GUI page. To address this issue, we propose Neat, a non-intrusive end-to-end mobile app layout similarity measurement tool that utilizes computer vision techniques for GUI element detection, layout feature extraction, and similarity metrics. Our empirical evaluation and industrial application have demonstrated that our approach is effective in improving the efficiency of layout assertion testing and ensuring application quality.


  • Open Access

    Hawkeye: Change-targeted Testing for Android Apps based on Deep Reinforcement Learning

    by Chao Peng, Zhengwei Lv, Jiarong Fu, Jiayuan Liang, Zhao Zhang, Ajitha Rajan, Ping Yang

    Android Apps are frequently updated to keep up with changing user, hardware, and business demands. Ensuring the correctness of App updates through extensive testing is crucial to avoid potential bugs reaching the end user. Existing Android testing tools generate GUI events that focus on improving the test coverage of the entire App rather than prioritising updates and impacted elements. Recent research has proposed change-focused testing but relies on random exploration to exercise change-impacted GUI elements that is ineffective and slow for large complex Apps with a huge input exploration space. At ByteDance, our established model-based GUI testing tool, Fastbot2, has been in successful deployment for nearly three years. Fastbot2 leverages event-activity transition models derived from past explorations to achieve enhanced test coverage efficiently. A pivotal insight we gained is that the knowledge of event-activity transitions is equally valuable in effectively targeting changes introduced by updates. This insight propelled our proposal for directed testing of updates with Hawkeye. Hawkeye excels in prioritizing GUI actions associated with code changes through deep reinforcement learning from historical exploration data. In our empirical evaluation, we rigorously compared Hawkeye with state-of-the-art tools like Fastbot2 and ARES on 10 popular open-source Apps and a commercial App. The results showcased that Hawkeye consistently outperforms Fastbot2 and ARES in generating GUI event sequences that effectively target changed functions, both in open-source and commercial App contexts. In real-world industrial deployment, Hawkeye is seamlessly integrated into our development pipeline, performing smoke testing for merge requests in a complex commercial App. The positive feedback received from our App development teams further affirmed Hawkeye's ability in testing App updates effectively.


  • Open Access

    AG3: Automated Game GUI Text Glitch Detection Based on Computer Vision

    by Xiaoyun Liang, Jiayi Qi, Yongqiang Gao, Chao Peng, Ping Yang

    With the advancement of device software and hardware performance, and the evolution of game engines, an increasing number of emerging high-quality games are captivating game players from all around the world who speak different languages. However, due to the vast fragmentation of the device and platform market, a well-tested game may still experience text glitches when installed on a new device with an unseen screen resolution and system version, which can significantly impact the user experience. In our testing pipeline, current testing techniques for identifying multilingual text glitches are laborious and inefficient. In this paper, we present AG3, which offers intelligent game traversal, precise visual text glitch detection, and integrated quality report generation capabilities. Our empirical evaluation and internal industrial deployment demonstrate that AG3 can detect various real-world multilingual text glitches with minimal human involvement.


  • Open Access

    Virtual Device Farms for Mobile App Testing at Scale: A Pursuit for Fidelity, Efficiency, and Accessibility

    by Hao Lin, Jiaxing Qiu, Hongyi Wang, Zhenhua Li, Liangyi Gong, Di Gao, Yunhao Liu, Feng Qian, Zhao Zhang, Ping Yang, et al.

    Virtual devices based on device emulation have been widely used in lab research of mobile app testing for their efficiency and low cost. However, it remains controversial to use virtual devices for app testing in industry, given the inherent difficulties of high-fidelity emulation across diverse mobile systems and devices. Hence, mobile app companies still rely on physical device farms or services like AWS Device Farm. This paper presents our effort to analyze, improve, and effectively use virtual devices for large-scale testing of mobile apps like Douyin. Our goal is to understand the fidelity of virtual devices and to explore how to better utilize virtual devices to improve the efficiency and accessibility of mobile app testing in industrial settings. Our study is conducted on a massive commercial testing infrastructure that deploys a physical device farm and its virtualized counterpart. We show that high-fidelity app testing can be achieved by sensible design and implementation of virtual device farms. With that, we find that major discrepancies are no longer caused by commonly believed factors like hardware heterogeneity and system customizations, but due to non-standard, uncoordinated, and occasionally defective vendor-specific services and drivers, as well as defense mechanisms against malicious app behavior. We present effective solutions to address those problems to significantly improve testing fidelity. We also share our experiences of using virtual device farms to substantially improve the efficiency and accessibility of mobile app testing, without compromising safety.