HSHanyuan Shi

Hanyuan Shi

*This is an algorithmically generated author record based on publicly available information.

Hanyuan Shi has a total of 15 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    ScratchNet: A Multi-modal Benchmark for Evaluating and Advancing LLMs on Scratch Programming Tasks

    by Yuan Si, Simeng Han, Daming Li, Hanyuan Shi, Jialu Zhang

    Large language models (LLMs) have achieved impressive performance on text-based programming tasks, yet they remain unreliable for block-based languages such as Scratch. Scratch programs feature deeply nested, nonlinear structures, event-driven concurrency across multiple sprites, and tight coupling between code and multimedia assets---properties that differ fundamentally from textual code. Consequently, LLMs frequently misinterpret Scratch semantics and propose large, invasive edits that are syntactically valid but semantically misaligned when repairing buggy programs. We introduce ScratchNet, the first executable benchmark designed to systematically evaluate and advance LLM-based repair for Scratch programs. The benchmark comprises 100 carefully curated projects from the public Scratch repository, each selected for high structural and semantic complexity. Every project is paired with an executable test suite, a bug description and corresponding fix, block-level edit constraints that define a minimal semantically correct repair, and the multimedia assets required for faithful execution. We construct the benchmark through a human-in-the-loop pipeline that combines automated project mining with expert validation of trigger--mechanism--outcome semantics and representative bug patterns, with particular emphasis on event ordering, concurrency, and state management. To enable rigorous and reproducible evaluation, we propose a three-layer executable protocol that measures functional correctness through VM-level execution, repair quality through block-level edit distance and behavioral trajectory comparisons, and explanation quality through structured rubrics. Using this benchmark, we study project and bug understanding, trigger and mechanism identification, functional repair, and the effect of lightweight domain adaptation. ScratchNet establishes a reproducible foundation and a closed-loop framework for evaluating and post-training LLMs on block-based programming tasks.

  • Open Access

    Context-Aware Feedback Compression in Online Judge Programming with LLMs

    by Jialiang Gu, Keren Zhou, Daming Li, Hanyuan Shi, Jialu Zhang

    Large language models (LLMs) can generate non-trivial programs, yet their reasoning often remains ungrounded: without external verification signals, one-shot generation may drift, repeat failure modes, or overfit to examples. We argue that the missing piece is budgeted feedback compression: turning noisy oracle outputs into compact, actionable hints that reliably drive multi-round code revision under tight context limits. We instantiate this idea in online judge (OJ) style algorithmic programming as a modular interactive agent that couples an LLM core with a sandboxed judger, a feedback-to-hint prompt constructor, and trajectory memory (optional error classifier). The key mechanism is feedback compression: converting noisy execution artifacts into compact, actionable hints within a tight prompt budget. In preliminary experiments, execution-grounded iteration improves debugging success from 83.9% (one-shot) to 93.2% on 570 real failed Codeforces submissions, and yields a clear difficulty trend in solving across 50 problems. Beyond OJ tasks, we envision budgeted oracle-to-hint compression as a general foundation for software engineering assistants that interact with continuous integration (CI) pipelines, tests, and profilers, shifting evaluation from final accuracy toward process metrics such as convergence and patch locality.

  • Open Access

    VisionScratch: LLM-Based Automated Feedback Generation using Code-Produced Videos for Scratch Programs

    by Yuan Si, Daming Li, Hanyuan Shi, Jialu Zhang

    Block-based programming environments such as Scratch are increasingly popular in programming education, in particular for young learners. While the use of blocks helps prevent syntax errors, semantic bugs remain common and difficult to debug. Existing tools for Scratch debugging rely heavily on predefined rules or user manual inputs, and crucially, they ignore the platform’s inherently visual nature. We introduce VisionScratch, the first multimodal feedback generation system for Scratch that leverages both the project’s block code and its generated gameplay video to diagnose and repair bugs. VisionScratch uses a two-stage pipeline: a vision-language model first aligns visual symptoms with code structure to identify a single critical issue, then proposes minimal, abstract syntax tree level repairs that are verified via execution in the Scratch virtual machine. We evaluate VisionScratch on a set of real-world Scratch projects against state-of-the-art LLM-based tools and human testers. Results show that gameplay video is a crucial debugging signal: VisionScratch substantially outperforms prior tools in both bug identification and repair quality, even without access to project descriptions or goals. This work demonstrates that video can serve as a first-class specification in visual programming environments, opening new directions for LLM-based debugging beyond symbolic code alone.


  • Open Access

    CBLab: Supporting the Training of Large-scale Traffic Control Policies with Scalable Traffic Simulation

    by Chumeng Liang, Zherui Huang, Yicheng Liu, Zhanyu Liu, Guanjie Zheng, Hanyuan Shi, Kan Wu, Yuhao Du, Fuliang Li, Zhenhui Jessie Li

    Traffic simulation provides interactive data for the optimization of traffic control policies. However, existing traffic simulators are limited by their lack of scalability and shortage in input data, which prevents them from generating interactive data from traffic simulation in the scenarios of real large-scale city road networks. In this paper, we present City Brain Lab, a toolkit for scalable traffic simulation. CBLab consists of three components: CBEngine, CBData, and CBScenario. CBEngine is a highly efficient simulator supporting large-scale traffic simulation. CBData includes a traffic dataset with road network data of 100 cities all around the world. We also develop a pipeline to conduct a one-click transformation from raw road networks to input data of our traffic simulation. Combining CBEngine and CBData allows researchers to run scalable traffic simulations in the road network of real large-scale cities. Based on that, CBScenario implements an interactive environment and a benchmark for two scenarios of traffic control policies respectively, with which traffic control policies adaptable for large-scale urban traffic can be trained and tuned. To the best of our knowledge, CBLab is the first infrastructure supporting traffic control policy optimization in large-scale urban scenarios. CBLab has supported the City Brain Challenge @ KDD CUP 2021. The project is available on GitHub:~https://github.com/CityBrainLab/CityBrainLab.git.