Transferable polychromatic optical encoder for neural networks
Preprint
Abstract
Artificial neural networks (ANNs) have fundamentally transformed the field of computer vision, providing unprecedented performance. However, these ANNs for image processing demand substantial computational resources, often hindering real-time operation. In this paper, we demonstrate an optical encoder that can perform convolution simultaneously in three color channels during the image capture, effectively implementing several initial convolutional layers of a ANN. Such an optical encoding results in ~24,000 times reduction in computational operations, with a state-of-the art classification accuracy (~73.2%) in free-space optical system. In addition, our analog optical encoder, trained for CIFAR-10 data, can be transferred to the ImageNet subset, High-10, without any modifications, and still exhibits moderate accuracy. Our results evidence the potential of hybrid optical/digital computer vision system in which the optical frontend can pre-process an ambient scene to reduce the energy and latency of the whole computer vision system.Keywords
All Related Versions
This publication has 36 references indexed in Scilit:
- Optical neural network via loose neuron array and functional learningNature Communications, 2023
- All-optical image classification through unknown random diffusers using a single-pixel diffractive networkLight: Science & Applications, 2023
- Intelligent metasurface system for automatic tracking of moving targets and wireless communications based on computer visionNature Communications, 2023
- Pruning and quantization for deep neural network acceleration: A surveyNeurocomputing, 2021
- CNN-based encoder-decoder networks for salient object detection: A comprehensive review and recent advancesInformation Sciences, 2021
- Synaptic organization of visual space in primary visual cortexNature, 2017
- Drones and digital photogrammetry: from classifications to continuums for monitoring river habitat and hydromorphologyWIREs Water, 2017
- Understanding natural scenes: Contributions of image statisticsNeuroscience & Biobehavioral Reviews, 2017
- Parallel processing strategies of the primate visual systemNature Reviews Neuroscience, 2009
- Speed of processing in the human visual systemNature, 1996