Xiang Li
Research Subjects
Xiang Li has a total of 27 co-authors. Here are the top co-authors by number of shared publications:
- Shan Liu
Sichuan Provincial Key Laboratory for Human Disease Gene Study, Department of Medical Genetics, Sichuan Academy of Medical Sciences & Sichuan Provincial People's Hospital, University of Electronic Science and Technology of China, Chengdu, 610072, China
4 shared publications - Han Gao
Tencent America, LLC (United States)
2 shared publications - Chih-Wei Hsu
MediaTek Inc., Hsinchu, Taiwan
1 shared publication - Chun-Chi Chen
QUALCOMM Inc., San Diego, CA, USA
1 shared publication - Dapeng Oliver Wu
City University of Hong Kong
1 shared publication Han Huang
Qualcomm Technologies Inc., San Diego, CA, USA
1 shared publication
Latest Publications
- 7 December 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 271-275
Unified Fast Partitioning Algorithm for Intra and Inter Predictions in Versatile Video Coding
The Versatile Video Coding (VVC) standard adopts a more flexible partitioning structure beyond the High Efficiency Video Coding (HEVC) standard. The quadtree with nested multi-type tree (QTMT) partitioning structure greatly improves coding efficiency. Nevertheless, the typical recursive coding unit (CU) partitioning scheme causes a substantial increase in computational complexity at the encoder. In this paper, a unified fast algorithm is proposed for both intra and inter predictions, which makes use of various historical information of previously checked partitions. The proposed algorithm is implemented on top of the VVC reference software VTM-14.0. Experimental results show that the proposed algorithm achieves an average 42.47%, 40.46%, and 40.38% encoder runtime reduction with only 0.45%, 1.08%, and 1.18% Bjøntegaard delta bitrate loss (BDBR) in All Intra (AI), Random Access (RA), and Low Delay P (LDP) configurations, respectively.
- 7 December 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 271-275
Unified Fast Partitioning Algorithm for Intra and Inter Predictions in Versatile Video Coding
The Versatile Video Coding (VVC) standard adopts a more flexible partitioning structure beyond the High Efficiency Video Coding (HEVC) standard. The quadtree with nested multi-type tree (QTMT) partitioning structure greatly improves coding efficiency. Nevertheless, the typical recursive coding unit (CU) partitioning scheme causes a substantial increase in computational complexity at the encoder. In this paper, a unified fast algorithm is proposed for both intra and inter predictions, which makes use of various historical information of previously checked partitions. The proposed algorithm is implemented on top of the VVC reference software VTM-14.0. Experimental results show that the proposed algorithm achieves an average 42.47%, 40.46%, and 40.38% encoder runtime reduction with only 0.45%, 1.08%, and 1.18% Bjøntegaard delta bitrate loss (BDBR) in All Intra (AI), Random Access (RA), and Low Delay P (LDP) configurations, respectively.
- Open Access
- 30 July 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 31 (10) , 3848-3861
Motion Vector Coding and Block Merging in the Versatile Video Coding Standard
This paper overviews the motion vector coding and block merging techniques in the Versatile Video Coding (VVC) standard developed by the Joint Video Experts Team (JVET). In general, inter-prediction techniques in VVC can be classified into two major groups: “whole block-based inter prediction” and “subblock-based inter prediction”. In this paper, we focus on techniques for whole block-based inter prediction. As in its predecessor, High Efficiency Video Coding (HEVC), whole block-based inter prediction in VVC is represented by adaptive motion vector prediction (AMVP) mode or merge mode. Newly introduced features purely for AMVP mode include symmetric motion vector difference and adaptive motion vector resolution. The features purely for merge mode include pairwise average merge, merge with motion vector difference, combined inter-intra prediction and geometric partitioning mode. Coding tools such as history-based motion vector prediction and bidirectional prediction with coding unit weights can be applied on both AMVP mode and merge mode. This paper discusses the design principles and the implementation of the new inter-prediction methods. Using objective metrics, simulation results show that the methods overviewed in the paper can jointly achieve 6.2% and 4.7% BD-rate savings on average with the random access and low-delay configurations, respectively. Significant subjective picture quality improvements of some tools are also reported when comparing the resulting pictures at same bitrates.
- 30 July 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 31 (10) , 3848-3861
Open AccessMotion Vector Coding and Block Merging in the Versatile Video Coding Standard
This paper overviews the motion vector coding and block merging techniques in the Versatile Video Coding (VVC) standard developed by the Joint Video Experts Team (JVET). In general, inter-prediction techniques in VVC can be classified into two major groups: “whole block-based inter prediction” and “subblock-based inter prediction”. In this paper, we focus on techniques for whole block-based inter prediction. As in its predecessor, High Efficiency Video Coding (HEVC), whole block-based inter prediction in VVC is represented by adaptive motion vector prediction (AMVP) mode or merge mode. Newly introduced features purely for AMVP mode include symmetric motion vector difference and adaptive motion vector resolution. The features purely for merge mode include pairwise average merge, merge with motion vector difference, combined inter-intra prediction and geometric partitioning mode. Coding tools such as history-based motion vector prediction and bidirectional prediction with coding unit weights can be applied on both AMVP mode and merge mode. This paper discusses the design principles and the implementation of the new inter-prediction methods. Using objective metrics, simulation results show that the methods overviewed in the paper can jointly achieve 6.2% and 4.7% BD-rate savings on average with the random access and low-delay configurations, respectively. Significant subjective picture quality improvements of some tools are also reported when comparing the resulting pictures at same bitrates.
- Open Access
- 11 June 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 31 (10) , 3818-3833
Block Partitioning Structure in the VVC Standard
Versatile Video Coding (VVC) is the latest video coding standard jointly developed by ITU-T VCEG and ISO/IEC MPEG. In this paper, technical details and experimental results for the VVC block partitioning structure are provided. Among all the new technical aspects of VVC, the block partitioning structure is identified as one of the most substantial changes relative to the previous video coding standards and provides the most significant coding gains. The new partitioning structure is designed using a more flexible scheme. Each coding tree unit (CTU) is either treated as one coding unit or split into multiple coding units by one or more recursive quaternary tree partitions followed by one or more recursive multi-type tree splits. The latter can be horizontal binary tree split, vertical binary tree split, horizontal ternary tree split, or vertical ternary tree split. A CTU dual tree for intra-coded slices is described on top of the new block partitioning structure, allowing separate coding trees for luma and chroma. Also, a new way of handling picture boundaries is presented. Additionally, to reduce hardware decoder complexity, virtual pipeline data unit constraints are introduced, which forbid certain multi-type tree splits. Finally, a local dual tree is described, which reduces the number of small chroma intra blocks.
- 11 June 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 31 (10) , 3818-3833
Open AccessBlock Partitioning Structure in the VVC Standard
Versatile Video Coding (VVC) is the latest video coding standard jointly developed by ITU-T VCEG and ISO/IEC MPEG. In this paper, technical details and experimental results for the VVC block partitioning structure are provided. Among all the new technical aspects of VVC, the block partitioning structure is identified as one of the most substantial changes relative to the previous video coding standards and provides the most significant coding gains. The new partitioning structure is designed using a more flexible scheme. Each coding tree unit (CTU) is either treated as one coding unit or split into multiple coding units by one or more recursive quaternary tree partitions followed by one or more recursive multi-type tree splits. The latter can be horizontal binary tree split, vertical binary tree split, horizontal ternary tree split, or vertical ternary tree split. A CTU dual tree for intra-coded slices is described on top of the new block partitioning structure, allowing separate coding trees for luma and chroma. Also, a new way of handling picture boundaries is presented. Additionally, to reduce hardware decoder complexity, virtual pipeline data unit constraints are introduced, which forbid certain multi-type tree splits. Finally, a local dual tree is described, which reduces the number of small chroma intra blocks.
- 1 February 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 31 (12) , 4924-4937
A Novel Video Coding Strategy in HEVC for Object Detection
Occupying the most significant portion of global data traffic, video is being generated in almost every aspect of our life. Because of its huge volume, we are depending much more heavily on machine intelligence based analysis. In the meantime, video coding technology has been continuously improved for better compression efficiency. However, the state-of-the-art video coding standards, such as H.265/HEVC and versatile video coding (VVC), are still designed assuming that the compressed video will be watched by a human later. Such a design is not optimal when the compressed video will be used by computer vision applications. While the human visual system (HVS) is consistently sensitive to the content with high contrast, the impact of pixels on computer vision algorithms is task driven. For example, because of the different categories of objects used to train detection algorithms, the influence of the same image content on those detectors also varies. Therefore, human oriented video coding strategies may not be optimal when the compressed signal is further processed by algorithms, as the encoder is unaware of the task specific information. In this article, taking object detection as an example, we propose a novel video coding strategy for computer vision. By protecting the information according to its importance for an object detector rather than for the human visual system, our proposed method has the potential to achieve a better object detection performance with the same bandwidth. The main contributions of our paper are: 1) the modeling of the relationship between object detection accuracy and bit rate; 2) a back propagation based method to analyze the influence of each pixel on the detection of target objects; 3) an object detection oriented bit allocation and codec control parameter determination scheme; 4) an evaluation metric to compare the impact of video coding strategies on a given object detector over a predefined range of bit rate. Experimental results demonstrate that our proposed algorithm can better preserve the video content vital for object detection than state-of-the-art video coding schemes.
- 1 February 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 31 (12) , 4924-4937
A Novel Video Coding Strategy in HEVC for Object Detection
Occupying the most significant portion of global data traffic, video is being generated in almost every aspect of our life. Because of its huge volume, we are depending much more heavily on machine intelligence based analysis. In the meantime, video coding technology has been continuously improved for better compression efficiency. However, the state-of-the-art video coding standards, such as H.265/HEVC and versatile video coding (VVC), are still designed assuming that the compressed video will be watched by a human later. Such a design is not optimal when the compressed video will be used by computer vision applications. While the human visual system (HVS) is consistently sensitive to the content with high contrast, the impact of pixels on computer vision algorithms is task driven. For example, because of the different categories of objects used to train detection algorithms, the influence of the same image content on those detectors also varies. Therefore, human oriented video coding strategies may not be optimal when the compressed signal is further processed by algorithms, as the encoder is unaware of the task specific information. In this article, taking object detection as an example, we propose a novel video coding strategy for computer vision. By protecting the information according to its importance for an object detector rather than for the human visual system, our proposed method has the potential to achieve a better object detection performance with the same bandwidth. The main contributions of our paper are: 1) the modeling of the relationship between object detection accuracy and bit rate; 2) a back propagation based method to analyze the influence of each pixel on the detection of target objects; 3) an object detection oriented bit allocation and codec control parameter determination scheme; 4) an evaluation metric to compare the impact of video coding strategies on a given object detector over a predefined range of bit rate. Experimental results demonstrate that our proposed algorithm can better preserve the video content vital for object detection than state-of-the-art video coding schemes.
- 28 February 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 31 (1) , 355-371
Fast DST-VII/DCT-VIII With Dual Implementation Support for Versatile Video Coding
The Joint Video Exploration Team (JVET) recently launched the standardization of the next-generation video coding named Versatile Video Coding (VVC) with the inherited technical framework from its predecessor High-Efficiency Video Coding (HEVC). The simplified Enhanced Multiple Transform (EMT) has been adopted as the primary residual coding transform solution, termed Multiple Transform Selection (MTS). In MTS, only the transform set consisting of DST-VII and DCT-VIII remains, excluding the other transform sets and the dependency on intra prediction modes. Significant coding gains are achieved by introducing new DST/DCT transforms, but the full matrix implementation is relatively costly compared to partial butterfly in terms of both software run-time and operation counts. In this work, we exploit the inherent features existing in DST-VII and DCT-VIII. Instead of repeating the element-wise additions and multiplications in full matrix operation, these features can be leveraged to achieve more efficient implementations which only use partial elements to derive the identical results. Existing transform matrices are further tuned to utilize these (anti-)symmetric features. A partial butterfly-type fast algorithm with dual-implementation support is proposed for DST-VII/DCT-VIII transform in VVC. Complexity analysis including operation counts and software run-time are conducted to validate the effectiveness. In addition, we prove the features are perfectly supported by theory. The proposed fast methods achieve noticeable software run-time savings without compromising on coding performance by comparing with the VVC Test Model VTM-3.0. It is shown that under Common Test Condition (CTC) with inter MTS enabled, an average of 9%, 0%, and 3% decoding time savings are achieved for All Intra (AI), Random Access (RA) and Low Delay B (LDB), respectively. Under low QP test condition with inter MTS enabled, the proposed fast methods achieve 1%, 2% and 4% decoding time savings on average for AI, RA, and LDB, respectively.
- 28 February 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 31 (1) , 355-371
Fast DST-VII/DCT-VIII With Dual Implementation Support for Versatile Video Coding
The Joint Video Exploration Team (JVET) recently launched the standardization of the next-generation video coding named Versatile Video Coding (VVC) with the inherited technical framework from its predecessor High-Efficiency Video Coding (HEVC). The simplified Enhanced Multiple Transform (EMT) has been adopted as the primary residual coding transform solution, termed Multiple Transform Selection (MTS). In MTS, only the transform set consisting of DST-VII and DCT-VIII remains, excluding the other transform sets and the dependency on intra prediction modes. Significant coding gains are achieved by introducing new DST/DCT transforms, but the full matrix implementation is relatively costly compared to partial butterfly in terms of both software run-time and operation counts. In this work, we exploit the inherent features existing in DST-VII and DCT-VIII. Instead of repeating the element-wise additions and multiplications in full matrix operation, these features can be leveraged to achieve more efficient implementations which only use partial elements to derive the identical results. Existing transform matrices are further tuned to utilize these (anti-)symmetric features. A partial butterfly-type fast algorithm with dual-implementation support is proposed for DST-VII/DCT-VIII transform in VVC. Complexity analysis including operation counts and software run-time are conducted to validate the effectiveness. In addition, we prove the features are perfectly supported by theory. The proposed fast methods achieve noticeable software run-time savings without compromising on coding performance by comparing with the VVC Test Model VTM-3.0. It is shown that under Common Test Condition (CTC) with inter MTS enabled, an average of 9%, 0%, and 3% decoding time savings are achieved for All Intra (AI), Random Access (RA) and Low Delay B (LDB), respectively. Under low QP test condition with inter MTS enabled, the proposed fast methods achieve 1%, 2% and 4% decoding time savings on average for AI, RA, and LDB, respectively.
- 1 November 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-5
Intra block copy in Versatile Video Coding with Reference Sample Memory Reuse
Screen contents such as online gaming streaming, remote desktop and WIFI display, become popular in current mainstream video applications. In versatile video coding (VVC), the most recent international video coding standard development, coding tools have been evaluated for optimizing screen content materials. Intra block copy (IBC) has shown its effectiveness in coding of computer-generated contents such as texts and graphics. Therefore, it has been previously included into the HEVC standard version 4, extensions for screen content coding (SCC). A constrained version of IBC mode has also been adopted in the VVC standard where the compensation range is limited within the current coding-tree unit (CTU), assuming a 1-CTU size of memory is allocated for storing IBC’s reference samples. In this paper, methods are proposed to efficiently utilize this reference sample memory for IBC mode such that effectively the search range for IBC mode can be increased without requiring more memory to store the reference samples. As a result, significant coding efficiency improvement over the traditional 1-CTU search range setting can be achieved. One of the proposed memory reuse strategies is considered practical for implementation and therefore has been included in the VVC standard.
- 1 November 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-5
Intra block copy in Versatile Video Coding with Reference Sample Memory Reuse
Screen contents such as online gaming streaming, remote desktop and WIFI display, become popular in current mainstream video applications. In versatile video coding (VVC), the most recent international video coding standard development, coding tools have been evaluated for optimizing screen content materials. Intra block copy (IBC) has shown its effectiveness in coding of computer-generated contents such as texts and graphics. Therefore, it has been previously included into the HEVC standard version 4, extensions for screen content coding (SCC). A constrained version of IBC mode has also been adopted in the VVC standard where the compensation range is limited within the current coding-tree unit (CTU), assuming a 1-CTU size of memory is allocated for storing IBC’s reference samples. In this paper, methods are proposed to efficiently utilize this reference sample memory for IBC mode such that effectively the search range for IBC mode can be increased without requiring more memory to store the reference samples. As a result, significant coding efficiency improvement over the traditional 1-CTU search range setting can be achieved. One of the proposed memory reuse strategies is considered practical for implementation and therefore has been included in the VVC standard.