Receptive Field Expanded Look-Up Tables for Vision Inference: Advancing from Low-level to High-level Tasks
Preprint
Abstract
Recently, several look-up table (LUT) methods were developed to greatly expedite the inference of CNNs in a classical strategy of trading space for speed. However, these LUT methods suffer from a common drawback of limited receptive field of the convolution kernels due to the combinatorial explosion of table size. This research aims to expand the CNN receptive field with a fixed table size, thereby enhancing the performance of LUT-driven fast CNN inference while maintaining the same space complexity. To achieve this goal, various techniques are proposed. The main contribution is a novel approach of learning an optimal lattice vector quantizer that adaptively allocates the quantization resolution across data dimensions based on their significance to the inference task. In addition, the lattice vector quantizer offers an inherently more accurate approximation of CNN kernels than scalar quantizer as used in current practice. Furthermore, we introduce other receptive field expansion strategies, including irregular dilated convolutions and a U-shaped cascaded LUT structure, designed to capture multi-level contextual information without inflating table size. Together, these innovations allow our approach to effectively balance speed, accuracy, and memory efficiency, demonstrating significant improvements over existing LUT methods.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- nnFormer: Volumetric Medical Image Segmentation via a 3D TransformerIEEE Transactions on Image Processing, 2023
- MuLUT: Cooperating Multiple Look-Up Tables for Efficient Image Super-Resolution2022
- Highly Efficient Salient Object Detection with 100K Parameters2020
- ResUNet-a: A deep learning framework for semantic segmentation of remotely sensed dataISPRS Journal of Photogrammetry and Remote Sensing, 2020
- Nucleus segmentation across imaging experiments: the 2018 Data Science BowlNature Methods, 2019
- Visual saliency based on multiscale deep features2015
- Progressive color transfer for images of arbitrary dynamic rangeComputers & Graphics, 2011
- The hardness of the closest vector problem with preprocessingIEEE Transactions on Information Theory, 2001
- Normalized cuts and image segmentationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2000
- Vector Quantization and Signal Compression1992