Jun Tang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Jun Tang has a total of 18 co-authors. Here are the top co-authors by number of shared publications:
- Yong Ding
Zhejiang University
5 shared publications - Linlin Liu
College of Integrated Circuits, Zhejiang University, Hangzhou, China
3 shared publications - Hang Dai
School of Computing Science, University of Glasgow, Glasgow, U.K.
2 shared publications Qing Yu
2 shared publications- Yifan Wang
College of Integrated Circuits, Zhejiang University, Hangzhou, China
2 shared publications - Yize Wang
College of Integrated Circuits, Zhejiang University, Hangzhou, China
2 shared publications
Latest Publications
- 22 October 2026
- Published by Elsevier in Computer Vision and Image Understanding
- Vol. 273
FreqSR: Frequency-aware network for Single-Image Super-Resolution
- 22 October 2026
- Published by Elsevier in Computer Vision and Image Understanding
- Vol. 273
FreqSR: Frequency-aware network for Single-Image Super-Resolution
- 14 July 2026
- Published by Elsevier in Pattern Recognition Letters
- Vol. 208, 63-68
O3DAA: A method for offline 3D point cloud automatic annotation
- 14 July 2026
- Published by Elsevier in Pattern Recognition Letters
- Vol. 208, 63-68
O3DAA: A method for offline 3D point cloud automatic annotation
- 2 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Broadcasting
- Vol. 72 (1) , 329-344
DM-VSR: Depth-Aware Diffusion Models With Adaptive Modulation for Video Super-Resolution
Video Super-Resolution (VSR) is essential for enhancing the quality of low-resolution (LR) videos in practical applications. Recent studies have explored diffusion models (DMs) for VSR due to their ability to generate realistic details. However, existing methods overlook spatiotemporal object-scale variations and dynamic control demands during denoising. This leads to visual distortion and quality degradation, severely limiting its practical applications. To address these limitations, we propose DM-VSR, a novel DM-based framework that incorporates depth-aware guidance and adaptive modulation for precise content reconstruction. Specifically, a Depth-aware Multimodal Fusion (DMF) module integrates depth maps, LR inputs, and flow-warped frames to provide unified depth-aware guidance. A Timestep Adaptive Modulation (TAM) module dynamically adjusts the control feature injection according to demand at each denoising step. Additionally, a Dynamic Consistency Loss (DCL) is introduced to align training objectives with the evolving semantic focus. Extensive experiments on the REDS4 and Vid4 benchmarks demonstrate that DM-VSR achieves competitive performance, surpassing state-of-the-art methods in both perceptual quality and temporal consistency. Moreover, DM-VSR generates more visually realistic results, emphasizing its effectiveness in real-world applications. The code will be released at https://github.com/aigcvsr/DM-VSR
- 2 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Broadcasting
- Vol. 72 (1) , 329-344
DM-VSR: Depth-Aware Diffusion Models With Adaptive Modulation for Video Super-Resolution
Video Super-Resolution (VSR) is essential for enhancing the quality of low-resolution (LR) videos in practical applications. Recent studies have explored diffusion models (DMs) for VSR due to their ability to generate realistic details. However, existing methods overlook spatiotemporal object-scale variations and dynamic control demands during denoising. This leads to visual distortion and quality degradation, severely limiting its practical applications. To address these limitations, we propose DM-VSR, a novel DM-based framework that incorporates depth-aware guidance and adaptive modulation for precise content reconstruction. Specifically, a Depth-aware Multimodal Fusion (DMF) module integrates depth maps, LR inputs, and flow-warped frames to provide unified depth-aware guidance. A Timestep Adaptive Modulation (TAM) module dynamically adjusts the control feature injection according to demand at each denoising step. Additionally, a Dynamic Consistency Loss (DCL) is introduced to align training objectives with the evolving semantic focus. Extensive experiments on the REDS4 and Vid4 benchmarks demonstrate that DM-VSR achieves competitive performance, surpassing state-of-the-art methods in both perceptual quality and temporal consistency. Moreover, DM-VSR generates more visually realistic results, emphasizing its effectiveness in real-world applications. The code will be released at https://github.com/aigcvsr/DM-VSR
- Open Access
- 14 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 11447-11462
VSRDiff: Learning Inter-Frame Temporal Coherence in Diffusion Model for Video Super-Resolution
Video Super-Resolution (VSR) aims to reconstruct high-quality high-resolution (HR) videos from low-resolution (LR) inputs. Recent studies have explored diffusion models (DMs) for VSR by exploiting their generative priors to produce realistic details. However, the inherent randomness of diffusion models presents significant challenges for controlling content. In particular, current DM-based VSR methods often neglect inter-frame temporal coherence and reconstruction-oriented objectives, leading to visual distortion and temporal inconsistency. In this paper, we introduce VSRDiff, a DM-based framework for VSR that emphasizes inter-frame temporal coherence and adopts a novel reconstruction perspective. Specifically, the Inter-Frame Aggregation Guidance (IFAG) module is developed to learn contextual inter-frame aggregation guidance, alleviating visual distortion caused by the randomness of diffusion models. Furthermore, the Progressive Reconstruction Sampling (PRS) approach is employed to generate reconstruction-oriented latents, balancing fidelity and detail richness. Additionally, temporal consistency is enhanced through second-order bidirectional latent propagation using the Flow-guided Latent Correction (FLC) module. Extensive experiments on the REDS4 and Vid4 datasets demonstrate that VSRDiff achieves highly competitive VSR performance with more realistic details, surpassing existing state-of-the-art methods in both visual fidelity and temporal consistency. Specifically, VSRDiff achieves the best scores on the REDS4 dataset in LPIPS, DISTS, and NIQE, with values of 0.1137, 0.0445, and 2.970, respectively. The result will be released at https://github.com/aigcvsr/VSRDiff.
- 14 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 11447-11462
Open AccessVSRDiff: Learning Inter-Frame Temporal Coherence in Diffusion Model for Video Super-Resolution
Video Super-Resolution (VSR) aims to reconstruct high-quality high-resolution (HR) videos from low-resolution (LR) inputs. Recent studies have explored diffusion models (DMs) for VSR by exploiting their generative priors to produce realistic details. However, the inherent randomness of diffusion models presents significant challenges for controlling content. In particular, current DM-based VSR methods often neglect inter-frame temporal coherence and reconstruction-oriented objectives, leading to visual distortion and temporal inconsistency. In this paper, we introduce VSRDiff, a DM-based framework for VSR that emphasizes inter-frame temporal coherence and adopts a novel reconstruction perspective. Specifically, the Inter-Frame Aggregation Guidance (IFAG) module is developed to learn contextual inter-frame aggregation guidance, alleviating visual distortion caused by the randomness of diffusion models. Furthermore, the Progressive Reconstruction Sampling (PRS) approach is employed to generate reconstruction-oriented latents, balancing fidelity and detail richness. Additionally, temporal consistency is enhanced through second-order bidirectional latent propagation using the Flow-guided Latent Correction (FLC) module. Extensive experiments on the REDS4 and Vid4 datasets demonstrate that VSRDiff achieves highly competitive VSR performance with more realistic details, surpassing existing state-of-the-art methods in both visual fidelity and temporal consistency. Specifically, VSRDiff achieves the best scores on the REDS4 dataset in LPIPS, DISTS, and NIQE, with values of 0.1137, 0.0445, and 2.970, respectively. The result will be released at https://github.com/aigcvsr/VSRDiff.
- 21 November 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Broadcasting
- Vol. 71 (1) , 334-349
VMG: Rethinking U-Net Architecture for Video Super-Resolution
The U-Net architecture has exhibited significant efficacy across various vision tasks, yet its adaptation for Video Super-Resolution (VSR) remains underexplored. While the Video Restoration Transformer (VRT) introduced U-Net into the VSR domain, it poses challenges due to intricate design and substantial computational overhead. In this paper, we present VMG, a streamlined framework tailored for VSR. Through empirical analysis, we identify the crucial stages of the U-Net architecture contributing to performance enhancement in VSR tasks. Our optimized architecture substantially reduces model parameters and complexity while improving performance. Additionally, we introduce two key modules, namely the Gated MLP-like Mixer (GMM) and the Flow-Guided cross-attention Mixer (FGM), designed to enhance spatial and temporal feature aggregation. GMM dynamically encodes spatial correlations with linear complexity in space and time, and FGM leverages optical flow to capture motion variation and implement sparse attention to efficiently aggregate temporally related information. Extensive experiments demonstrate that VMG achieves nearly 70% reduction in GPU memory usage, 30% fewer parameters, and 10% lower computational complexity (FLOPs) compared to VRT, while yielding highly competitive or superior results across four benchmark datasets. Qualitative assessments reveal VMG’s ability to preserve remarkable details and sharp structures in the reconstructed videos. The code and pre-trained models are available at https://github.com/EasyVision-Ton/VMG.
- 21 November 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Broadcasting
- Vol. 71 (1) , 334-349
VMG: Rethinking U-Net Architecture for Video Super-Resolution
The U-Net architecture has exhibited significant efficacy across various vision tasks, yet its adaptation for Video Super-Resolution (VSR) remains underexplored. While the Video Restoration Transformer (VRT) introduced U-Net into the VSR domain, it poses challenges due to intricate design and substantial computational overhead. In this paper, we present VMG, a streamlined framework tailored for VSR. Through empirical analysis, we identify the crucial stages of the U-Net architecture contributing to performance enhancement in VSR tasks. Our optimized architecture substantially reduces model parameters and complexity while improving performance. Additionally, we introduce two key modules, namely the Gated MLP-like Mixer (GMM) and the Flow-Guided cross-attention Mixer (FGM), designed to enhance spatial and temporal feature aggregation. GMM dynamically encodes spatial correlations with linear complexity in space and time, and FGM leverages optical flow to capture motion variation and implement sparse attention to efficiently aggregate temporally related information. Extensive experiments demonstrate that VMG achieves nearly 70% reduction in GPU memory usage, 30% fewer parameters, and 10% lower computational complexity (FLOPs) compared to VRT, while yielding highly competitive or superior results across four benchmark datasets. Qualitative assessments reveal VMG’s ability to preserve remarkable details and sharp structures in the reconstructed videos. The code and pre-trained models are available at https://github.com/EasyVision-Ton/VMG.
- 7 December 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 34 (6) , 5018-5032
CTVSR: Collaborative Spatial–Temporal Transformer for Video Super-Resolution
Video super-resolution (VSR) is important in video processing for reconstructing high-definition image sequences from corresponding continuous and highly-related video frames. However, existing VSR methods have limitations in fusing spatial-temporal information. Some methods only fuse spatial-temporal information on a limited range of total input sequences, while others adopt a recurrent strategy that gradually attenuates the spatial information. While recent advances in VSR utilize Transformer-based methods to improve the quality of the upscaled videos, these methods require significant computational resources to model the long-range dependencies, which dramatically increases the model complexity. To address these issues, we propose a Collaborative Transformer for Video Super-Resolution (CTVSR). The proposed method integrates the strengths of Transformer-based and recurrent-based models by concurrently assimilating the spatial information derived from multi-scale receptive fields and the temporal information acquired from temporal trajectories. In particular, we propose a Spatial Enhanced Network (SEN) with two key components: Token Dropout Attention (TDA) and Deformable Multi-head Cross Attention (DMCA). TDA focuses on the key regions to extract more informative features, and DMCA employs deformable cross attention to gather information from adjacent frames. Moreover, we introduce a Temporal-trajectory Enhanced Network (TEN) that computes the similarity of a given token with temporally-related tokens in the temporal trajectory, which is different from previous methods that evaluate all tokens within the temporal dimension. With comprehensive quantitative and qualitative experiments on four widely-used VSR benchmarks, the proposed CTVSR achieves competitive performance with relatively low computational consumption and high forward speed.
- 7 December 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 34 (6) , 5018-5032
CTVSR: Collaborative Spatial–Temporal Transformer for Video Super-Resolution
Video super-resolution (VSR) is important in video processing for reconstructing high-definition image sequences from corresponding continuous and highly-related video frames. However, existing VSR methods have limitations in fusing spatial-temporal information. Some methods only fuse spatial-temporal information on a limited range of total input sequences, while others adopt a recurrent strategy that gradually attenuates the spatial information. While recent advances in VSR utilize Transformer-based methods to improve the quality of the upscaled videos, these methods require significant computational resources to model the long-range dependencies, which dramatically increases the model complexity. To address these issues, we propose a Collaborative Transformer for Video Super-Resolution (CTVSR). The proposed method integrates the strengths of Transformer-based and recurrent-based models by concurrently assimilating the spatial information derived from multi-scale receptive fields and the temporal information acquired from temporal trajectories. In particular, we propose a Spatial Enhanced Network (SEN) with two key components: Token Dropout Attention (TDA) and Deformable Multi-head Cross Attention (DMCA). TDA focuses on the key regions to extract more informative features, and DMCA employs deformable cross attention to gather information from adjacent frames. Moreover, we introduce a Temporal-trajectory Enhanced Network (TEN) that computes the similarity of a given token with temporally-related tokens in the temporal trajectory, which is different from previous methods that evaluate all tokens within the temporal dimension. With comprehensive quantitative and qualitative experiments on four widely-used VSR benchmarks, the proposed CTVSR achieves competitive performance with relatively low computational consumption and high forward speed.