Shiyi Zhang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Shiyi Zhang has a total of 13 co-authors. Here are the top co-authors by number of shared publications:
- Yansong Tang
Tsinghua Shenzhen International Graduate School, Tsinghua University
3 shared publications - Changliu Liu
Carnegie Mellon University, Pittsburgh, PA, USA
1 shared publication - Guanxing Lu
Tsinghua University
1 shared publication Jingqi Tian
Tsinghua Shenzhen International Graduate School Tsinghua University
1 shared publication- Jiwen Lu
Tsinghua University
1 shared publication - Junhao Zhuang
Tsinghua University, Hong Kong, China
1 shared publication
Latest Publications
- Open Access
Memorize-and-Generate: Towards Long-Term Consistency in Real-Time Video Generation
Frame-level autoregressive (frame-AR) models have achieved significant progress, enabling real-time video generation comparable to bidirectional diffusion models and serving as a foundation for interactive world models and game engines. However, current approaches in long video generation typically rely on window attention, which naively discards historical context outside the window, leading to catastrophic forgetting and scene inconsistency; conversely, retaining full history incurs prohibitive memory costs. To address this trade-off, we propose Memorize-and-Generate (MAG), a framework that decouples memory compression and frame generation into distinct tasks. Specifically, we train a memory model to compress historical information into a compact KV cache, and a separate generator model to synthesize subsequent frames utilizing this compressed representation. Furthermore, we introduce MAG-Bench to strictly evaluate historical memory retention. Extensive experiments demonstrate that MAG achieves superior historical scene consistency while maintaining competitive performance on standard video generation benchmarks.
Open AccessMemorize-and-Generate: Towards Long-Term Consistency in Real-Time Video Generation
Frame-level autoregressive (frame-AR) models have achieved significant progress, enabling real-time video generation comparable to bidirectional diffusion models and serving as a foundation for interactive world models and game engines. However, current approaches in long video generation typically rely on window attention, which naively discards historical context outside the window, leading to catastrophic forgetting and scene inconsistency; conversely, retaining full history incurs prohibitive memory costs. To address this trade-off, we propose Memorize-and-Generate (MAG), a framework that decouples memory compression and frame generation into distinct tasks. Specifically, we train a memory model to compress historical information into a compact KV cache, and a separate generator model to synthesize subsequent frames utilizing this compressed representation. Furthermore, we introduce MAG-Bench to strictly evaluate historical memory retention. Extensive experiments demonstrate that MAG achieves superior historical scene consistency while maintaining competitive performance on standard video generation benchmarks.
- Open Access
- 27 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-11
FlexiAct: Towards Flexible Action Control in Heterogeneous Scenarios
- 27 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-11
Open AccessFlexiAct: Towards Flexible Action Control in Heterogeneous Scenarios
- Top Cited Papers
- 30 September 2024
- Published by Springer Nature
- p. 349-366
ManiGaussian: Dynamic Gaussian Splatting for Multi-task Robotic Manipulation
- 30 September 2024
- Published by Springer Nature
- p. 349-366
Top Cited PapersManiGaussian: Dynamic Gaussian Splatting for Multi-task Robotic Manipulation
- 22 September 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 175-179
Research on Intelligent Sorting Technology Based on Neural Network
The garbage sorting work takes the community as the main battlefield, and the family as a social cell undertakes the primary task of garbage sorting. In order to solve the problem of difficult household waste sorting, we designed a device called “Neural Network-Based Intelligent Sorting Device”. The device uses Jetson Nano technology to achieve correct object recognition and classification by identifying objects captured by the camera. At the same time, we improved the YOLOv3 target detection algorithm, replaced the original feature extraction network with Mobile-NetV2 to reduce the feature scale, and applied the adjusted Focal loss loss function to the algorithm. Through prototype experiments, we found that the improved algorithm can significantly improve the detection accuracy and speed, thereby improving the efficiency and accuracy of garbage classification.
- 22 September 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 175-179
Research on Intelligent Sorting Technology Based on Neural Network
The garbage sorting work takes the community as the main battlefield, and the family as a social cell undertakes the primary task of garbage sorting. In order to solve the problem of difficult household waste sorting, we designed a device called “Neural Network-Based Intelligent Sorting Device”. The device uses Jetson Nano technology to achieve correct object recognition and classification by identifying objects captured by the camera. At the same time, we improved the YOLOv3 target detection algorithm, replaced the original feature extraction network with Mobile-NetV2 to reduce the feature scale, and applied the adjusted Focal loss loss function to the algorithm. Through prototype experiments, we found that the improved algorithm can significantly improve the detection accuracy and speed, thereby improving the efficiency and accuracy of garbage classification.