Scalable Image Coding for Humans and Machines
Top Cited Papers
- 24 March 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 31 (10577149) , 2739-2754
Abstract
At present, and increasingly so in the future, much of the captured visual content will not be seen by humans. Instead, it will be used for automated machine vision analytics and may require occasional human viewing. Examples of such applications include traffic monitoring, visual surveillance, autonomous navigation, and industrial machine vision. To address such requirements, we develop an end-to-end learned image codec whose latent space is designed to support scalability from simpler to more complicated tasks. The simplest task is assigned to a subset of the latent space (the base layer), while more complicated tasks make use of additional subsets of the latent space, i.e., both the base and enhancement layer(s). For the experiments, we establish a 2-layer and a 3-layer model, each of which offers input reconstruction for human vision, plus machine vision task(s), and compare them with relevant benchmarks. The experiments show that our scalable codecs offer 37%–80% bitrate savings on machine vision tasks compared to best alternatives, while being comparable to state-of-the-art image codecs in terms of input reconstruction.Keywords
All Related Versions
Funding Information
- Vanier Canada Graduate Scholarship
- NSERC under Discovery Grant (RGPIN-2021-02485, RGPAS-2021-00038)
This publication has 42 references indexed in Scilit:
- SSD: Single Shot MultiBox Detector2016
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksIEEE Transactions on Pattern Analysis and Machine Intelligence, 2016
- Deep Residual Learning for Image Recognition2016
- Overview of SHVC: Scalable Extensions of the High Efficiency Video Coding StandardIEEE Transactions on Circuits and Systems for Video Technology, 2015
- Deriving video content type from HEVC bitstream semantics2014
- Video Object Tracking in the Compressed Domain Using Spatio-Temporal Markov Random FieldsIEEE Transactions on Image Processing, 2012
- An Edge Detection Method in DCT DomainProcedia Engineering, 2012
- Distinctive Image Features from Scale-Invariant KeypointsInternational Journal of Computer Vision, 2004
- Multiscale structural similarity for image quality assessment2004
- Arithmetic coding for data compressionCommunications of the ACM, 1987