Survey on Activation Functions for Optical Neural Networks
Open Access
- 15 September 2023
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Computing Surveys
- Vol. 56 (2) , 1-30
Abstract
Integrated photonics arises as a fast and energy-efficient technology for the implementation of artificial neural networks (ANNs). Indeed, with the growing interest in ANNs, photonics shows great promise to overcome current limitations of electronic-based implementation. For example, it has been shown that neural networks integrating optical matrix multiplications can potentially run two orders of magnitude faster than their electronic counterparts. However, the transposition in the optical domain of the activation functions, which is a key feature of ANNs, remains a challenge. There is no direct optical implementation of state-of-the-art activation functions. Currently, most designs require time-consuming and power-hungry electro-optical conversions. In this survey, we review both all-optical and opto-electronic activation functions proposed in the state-of-the-art. We present activation functions with their key characteristics, and we summarize challenges for their use in the context of all-optical neural networks. We then highlight research directions for the implementation of fully optical neural networks.Keywords
This publication has 57 references indexed in Scilit:
- Nanophotonic Reservoir Computing With Photonic Crystal Cavities to Generate Periodic PatternsIEEE Transactions on Neural Networks and Learning Systems, 2013
- Mean Squared Error2011
- Monolayer graphene as a saturable absorber in a mode-locked laserNano Research, 2010
- Deep, Big, Simple Neural Nets for Handwritten Digit RecognitionNeural Computation, 2010
- Slow light on a chip via atomic quantum state controlNature Photonics, 2010
- Amorphous silicon balanced photodiode for detection of ultraviolet radiationSensors and Actuators A: Physical, 2009
- An experimental unification of reservoir computing methodsNeural Networks, 2007
- A computational model of filtering, detection, and compression in the cochlea2005
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- Self-phase modulation and spectral broadening of optical pulses in semiconductor laser amplifiersIEEE Journal of Quantum Electronics, 1989