Huibing Wang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Huibing Wang has a total of 7 co-authors. Here are the top co-authors by number of shared publications:
- Guangming Tan
Chinese Academy of Sciences
1 shared publication - Hui Ma
Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
1 shared publication - Junmin Xiao
Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
1 shared publication - Mingyi Li
Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
1 shared publication - Xiaochuan Deng
Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
1 shared publication - Yunfei Pang
Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
1 shared publication
Latest Publications
- Open Access
- 22 March 2026
- Published by Association for Computing Machinery (ACM)
- p. 1949-1965
T-Control: An Efficient Dynamic Tensor Rematerialization System for DNN Training
With the continuous growth of model and batch sizes, DNN model training increasingly suffers from excessive memory consumption. Tensor rematerialization has emerged as an effective technique to enable training under limited memory constraints. However, dynamic rematerialization methods often underperform static approaches, primarily due to their greedy tensor eviction schedules and runtime overhead. In this work, we develop T-Control, a dynamic tensor rematerialization system for DNN model training. T-Control integrates the topology of the traced tensor dependency graph with real-time memory usage to make informed, adaptive tensor retention decisions, preserving critical tensors and reducing eviction-induced recomputation. Furthermore, by extending PyTorch's native memory manager with our fine-grained memory management strategy, T-control effectively improves memory utilization, thereby reducing unnecessary tensor eviction. Experimental results demonstrate that T-Control boosts throughput by up to 1.58× and 1.91× over state-of-the-art static and dynamic tensor rematerialization systems.
- 22 March 2026
- Published by Association for Computing Machinery (ACM)
- p. 1949-1965
Open AccessT-Control: An Efficient Dynamic Tensor Rematerialization System for DNN Training
With the continuous growth of model and batch sizes, DNN model training increasingly suffers from excessive memory consumption. Tensor rematerialization has emerged as an effective technique to enable training under limited memory constraints. However, dynamic rematerialization methods often underperform static approaches, primarily due to their greedy tensor eviction schedules and runtime overhead. In this work, we develop T-Control, a dynamic tensor rematerialization system for DNN model training. T-Control integrates the topology of the traced tensor dependency graph with real-time memory usage to make informed, adaptive tensor retention decisions, preserving critical tensors and reducing eviction-induced recomputation. Furthermore, by extending PyTorch's native memory manager with our fine-grained memory management strategy, T-control effectively improves memory utilization, thereby reducing unnecessary tensor eviction. Experimental results demonstrate that T-Control boosts throughput by up to 1.58× and 1.91× over state-of-the-art static and dynamic tensor rematerialization systems.