FLFan Liang

Fan Liang

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Fan Liang has a total of 25 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Hierarchical Distortion Learning for Fast Lossy Compression of Point Clouds

    by Pengpeng Yu, Ye Zhang, Fan Liang, Haoran Li, Yulan Guo

    The growth of 3D point cloud applications requires efficient compression techniques for high-quality and low-latency services. Recently, learning-based point cloud compression models have made significant progress. However, geometric distortion resulting from downsampling limits the feature depth within large-scale point clouds, thereby constraining the receptive field and suppressing the redundant removal. Moreover, the issues of computational efficiency and reconstruction quality still persist in the compression of large-scale point clouds. To address these challenges, we propose a hierarchical distortion learning framework for end-to-end lossy compression of point clouds. First, we design a feature residual compression module to efficiently transmit shallow semantics between the encoder and the decoder, which enables a lightweight design of our framework. Second, we introduce a geometry residual compression module to progressively complement the reconstruction distortion, avoiding the accumulation of geometric distortion. By integrating these two modules and employing sufficient downsampling processes, we develop a high-performance framework with a significantly enlarged receptive field and low computational cost. Extensive experiments demonstrate that our method achieves state-ofthe- art performance in geometry lossy compression, while delivering competitive performance in joint geometry and color lossy compression with fast running speed. Code is available at https://github.com/pengpeng-yu/FastPCC.


  • Encounter CU Again: History-Based Complexity Reduction Strategy for VVC Intra-Frame Encoder

    by Jian Cao, Yifan Jia, Fan Liang, Jun Wang

    Due to the newly adopted Quad Tree with Nested Multi-Type Tree (QTMT) partitioning scheme in Versatile Video Coding (VVC), multiple partitioning combinations can lead to the same Coding Unit (CU) structure. In other words, a CU may be encoded more than once. Based on this feature, a history-based complexity reduction strategy is proposed to accelerate VVC intra-frame coding with extremely low coding losses.Firstly, analyses of the relationship between the 1 st round CUs (encoded at the first time) and the following rounds CUs (encountered again and already analyzed in previous partitioning attempts) are provided. Correspondingly, some unnecessary partitioning types are identified and early terminated. Secondly, a hierarchical pruning algorithm is designed, where thresholds are adjusted adaptively in the 1 st round and used for pruning in the following rounds. To our knowledge, it is the first attempt to apply this history-based feature to accelerate partitioning for VVC intra-frame coding.Results show that these strategies can achieve 20% encoding time saving (TS) with only 0.18% BDBR increase. In addition, there is a huge potential for High-Resolution videos (21% TS with only 0.1% BDBR increase for 4K sequences). Compared to other works, our method achieves a considerably high TS/BDBR ratio, which indicates a better tradeoff between coding efficiency and complexity.


  • Open Access

    Maria: A Visual Experience Powered Conversational Agent

    by Zujie Liang, Huang Hu, Can Xu, Chongyang Tao, Xiubo Geng, Yining Chen, Fan Liang, Daxin Jiang

    Arguably, the visual perception of conversational agents to the physical world is a key way for them to exhibit the human-like intelligence. Image-grounded conversation is thus proposed to address this challenge. Existing works focus on exploring the multimodal dialog models that ground the conversation on a given image. In this paper, we take a step further to study image-grounded conversation under a fully open-ended setting where no paired dialog and image are assumed available. Specifically, we present Maria, a neural conversation agent powered by the visual world experiences which are retrieved from a large-scale image index. Maria consists of three flexible components, i.e., text-to-image retriever, visual concept detector and visual-knowledge-grounded response generator. The retriever aims to retrieve a correlated image to the dialog from an image index, while the visual concept detector extracts rich visual knowledge from the image. Then, the response generator is grounded on the extracted visual knowledge and dialog context to generate the target response. Extensive experiments demonstrate Maria outperforms previous state-of-the-art methods on automatic metrics and human evaluation, and can generate informative responses that have some visual commonsense of the physical world.

  • Open Access

    Intra Block Copy Mirror Mode for Screen Content Coding in Versatile Video Coding

    by Jian Cao, Fan Liang, Jun Wang

    Intra Block Copy (IBC) cannot work well for blocks with symmetry or mirror characteristic, because it is based on the translational block matching technique. In order to solve this problem, in this paper, an adequately designed IBC-Mirror mode is proposed for Versatile Video Coding (VVC), which extends our previous work. The newly proposed IBC-Mirror mode consists of two sub-modes, including IBC-Mirror Advanced Motion Vector Prediction (AMVP) mode and IBC-Mirror Merge mode. In IBC-Mirror AMVP mode, various Motion Estimation (ME) methods are applied, including previously proposed spatial search, and newly designed methods, i.e., Hash-based, Cache-based, Predictor-based and History-based IBC-Mirror search. As for IBC-Mirror Merge mode, Motion Information, composed of Block Vector (BV) and mirror flipping information, is added to the merge list to improve coding efficiency. Compared to the traditional IBC mode in VTM-10.0, the latest test model, the proposed mode can achieve 1~2% BD-rate saving. Compared to our previous work, methods in this paper can further achieve 0.5% coding gains and reduce 4% coding time. Higher coding efficiency and lower complexity show the superiority of extensions and improvements in this paper.


  • Open Access

    A GAN-Based Video Intra Coding

    by Guangyu Zhong, Jun Wang, Jiyuan Hu, Fan Liang

    Intra prediction is a vital part of the image/video coding framework, which is designed to remove spatial redundancy within a picture. Based on a set of predefined linear combinations, traditional intra prediction cannot cope with coding blocks with irregular textures. To tackle this drawback, in this article, we propose a Generative Adversarial Network (GAN)-based intra prediction approach to enhance intra prediction accuracy. Specifically, with the superior non-linear fitting ability, the well-trained generator of GAN acts as a mapping from the adjacent reconstructed signals to the prediction unit, implemented into both encoder and decoder. Simulation results show that for All-Intra configuration, our proposed algorithm achieves, on average, a 1.6% BD-rate cutback for luminance components compared with video coding reference software HM-16.15 and outperforms previous similar works.


  • Feature based inter prediction optimization for non-translational video coding in cloud

    by Xuelin Shen, Jun Wang, Qing Yang, Peilin Chen, Fan Liang

    Visual features of images and video frames have become pervasive and maturely developed in extensive research fields such as computer vision and visual search. In more and more cases, the visual feature becomes necessary information which needs to be transmitted and stored at server side in cloud. Among visual features, the local feature descriptors extracted by SIFT can represent both translational and non-translational motion, such as orientation and zooming. On the other hand, only translational motion can be represented by the Motion Vector (MV) in current MV based block video coding standard. Inspired by these properties, a method that utilizes the available feature to optimize inter prediction video coding is proposed in this paper. In this method, the localization, orientation and scale parameters of matching features extracted by SIFT are delivered to inter prediction to provide non-translational motion estimation (ME) and optimized merge mode. Experimental results have shown that the proposed method can efficiently improve the coding performance according to the accurate feature-matching.


  • A DWT-Based Video Watermarking Algorithm Applying DS-CDMA

    by Liang Fan, Fang Yanmei

    In this paper, we propose a DWT-based video watermarking algorithm applying DS-CDMA (direct sequence code division multiple access). We use improved Gold sequences to encode the watermark. The CDMA encoded watermark is embedded into the lowest frequency coefficients of DWT transformed video frames. Blind recovery of the embedded watermark is achieved by analyzing the DWT coefficients of the watermarked video frames and the auto-correlation of orthogonal codes. The experimental results demonstrate that the proposed algorithm can produce watermarked contents in good visual quality and is robust to typical video operations such as MPEG-2 compression, Gaussian noise, cropping and frame dropping