DQDong Qiang
Dong Qiang
*This is an algorithmically generated author record based on publicly available information.
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Dong Qiang has a total of 5 co-authors. Here are the top co-authors by number of shared publications:
- Changhao Ma
Ningxia University, Yinchuan, China
1 shared publication - Hao Xu
College of Electronics and Information Engineering, Tongji University, Shanghai, China
1 shared publication - Jiawei Dai
Ningxia University, Yinchuan, China
1 shared publication - Jie Yang
Institute of Ethnic Preparatory Education, Yinchuan, China and Ningxia University, Yinchuan, China
1 shared publication - Suping Wu
Ningxia University
1 shared publication
Latest Publications
- Open Access
- 27 September 2026
- Published by Association for Computing Machinery (ACM)
- p. 920-930
Async-Checkpoint: Redesigning Gradient Checkpointing with Asynchronous Execution
Activation recomputation (gradient checkpointing) reduces the memory footprint of intermediate activations from O(n) to \(O(\sqrt {n})\) by selectively discarding and recomputing them during the backward pass. However, a fundamental limitation persists: all existing implementations follow a synchronous blocking paradigm, forcing recomputation to fully occupy the critical path of backpropagation and typically introducing 30%–40% additional training overhead. This paper proposes Async-Checkpoint, an asynchronous activation recomputation algorithm. Our core insight is that the serial dependency between the recomputation and gradient computation of adjacent blocks is not required by data flow. Based on this, we redesign the execution logic—replacing the serial “recompute-then-gradient” loop with an asynchronous pipeline that overlaps the two tasks in separate CUDA streams, thereby removing recomputation from the critical path. Experiments on BERT, ResNet, U-Net, and a recurrent MVS model demonstrate generality across linear chains, DAGs with skip connections, and dynamic recurrent structures. Async-Checkpoint consistently reduces iteration time compared to the PyTorch synchronous implementation while maintaining memory consumption on par with synchronous checkpointing.- 27 September 2026
- Published by Association for Computing Machinery (ACM)
- p. 920-930
Open AccessAsync-Checkpoint: Redesigning Gradient Checkpointing with Asynchronous Execution
Activation recomputation (gradient checkpointing) reduces the memory footprint of intermediate activations from O(n) to \(O(\sqrt {n})\) by selectively discarding and recomputing them during the backward pass. However, a fundamental limitation persists: all existing implementations follow a synchronous blocking paradigm, forcing recomputation to fully occupy the critical path of backpropagation and typically introducing 30%–40% additional training overhead. This paper proposes Async-Checkpoint, an asynchronous activation recomputation algorithm. Our core insight is that the serial dependency between the recomputation and gradient computation of adjacent blocks is not required by data flow. Based on this, we redesign the execution logic—replacing the serial “recompute-then-gradient” loop with an asynchronous pipeline that overlaps the two tasks in separate CUDA streams, thereby removing recomputation from the critical path. Experiments on BERT, ResNet, U-Net, and a recurrent MVS model demonstrate generality across linear chains, DAGs with skip connections, and dynamic recurrent structures. Async-Checkpoint consistently reduces iteration time compared to the PyTorch synchronous implementation while maintaining memory consumption on par with synchronous checkpointing.