Variable-Rate Deep Image Compression With Vision Transformers
Open Access
- 9 May 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 10, 50323-50334
Abstract
Recently, vision transformers have been applied in many computer vision problems due to its long-range learning ability. However, it has not been throughly explored in image compression. We propose a patch-based learned image compression network by incorporating vision transformers. The input image is divided into patches before feeding to the encoder and the patches are reconstructed from the decoder to form a complete image. Different kinds of transformer blocks (TransBlocks) are applied to meet the various requirements in the subnetworks. We also propose a transformer-based context model (TransContext) to facilitate the coding based on previously decoded symbols. Since the computational complexity of the attention mechanism in transformers is a quadratic function of the sequence length, we partition the feature tensor into different segments and conduct the transformer in each segment to save computational cost. To alleviate the compression artifacts, we use overlapping patches and apply an existing deblocking network to further remove the artifacts. At last, the residual coding scheme is adopted to get the compression performance for variable bit rates. We show that our patch-based learned image compression with transformers obtain 0.75dB improvement in PSNR at 0.15bpp than the prior variable-rate compression work on the Kodak dataset. When using the residual coding strategy, our framework keeps good performance in PSNR and is comparable to BPG420. For MS-SSIM, we get higher results than BPG444 across a range of bit rates (0.021 at 0.21bpp) and other variable-rate learned image compression models at low bit rates.Keywords
All Related Versions
Funding Information
- Natural Sciences and Engineering Research Council of Canada (RGPIN-2020-04525)
- Google Chrome University Research Program
This publication has 31 references indexed in Scilit:
- Efficient Variable Rate Image Compression With Multi-Scale Decomposition NetworkIEEE Transactions on Circuits and Systems for Video Technology, 2018
- Fast, Accurate, and Lightweight Super-Resolution with Cascading Residual Network2018
- Deep Residual Learning for Image Recognition2016
- Compression Artifacts Reduction by a Deep Convolutional Network2015
- Information technology. JPEG 2000 image coding system2015
- Microsoft COCO: Common Objects in Context2014
- Inpainting with image patches for compressionJournal of Visual Communication and Image Representation, 2012
- Multiscale structural similarity for image quality assessment2004
- A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics2002
- The JPEG still picture compression standardCommunications of the ACM, 1991