Pseudoinvertible Neural Networks
- 18 July 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Artificial Intelligence
- Vol. 5 (2) , 602-612
Abstract
This work introduces novel architecture components and training procedures to create augmented neural networks with the ability to process data bidirectionally via an end-to-end approximate inverse. We develop Pseudo-Invertible Neural Network (PsI-NN) layers which function as drop-in replacements for corresponding convolutional and fully connected layers; by using these, existing architectures gain a pseudoinverse function which, with training, approximately reverses the forward function. For cases where learning both a task and its inverse are necessary or desirable, we show that PsI-NN enabled models match or exceed the quality of results generated by systems that use two separate models while drastically reducing system parameter count. We demonstrate this on two tasks: unpaired image translation and semi-supervised classification. In both, PsI-NN greatly reduces parameter count without any loss of output quality; in semi supervised image classification, PsI-NN improves classification performance beyond the baseline autoencoder method. Our approach creates pseudo-invertible versions of existing architectures, circumventing the stringent constraints required to create a true inverse while allowing a single neural network to learn a task in two directions simultaneously.Keywords
This publication has 12 references indexed in Scilit:
- Restorable Image Operators with Quasi-Invertible NetworksProceedings of the AAAI Conference on Artificial Intelligence, 2022
- Reversible Architectures for Arbitrarily Deep Residual Neural NetworksProceedings of the AAAI Conference on Artificial Intelligence, 2018
- Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks2017
- Deep Convolutional Neural Network for Inverse Problems in ImagingIEEE Transactions on Image Processing, 2017
- Autoencoder With Invertible Functions for Dimension Reduction and Image ReconstructionIEEE Transactions on Systems, Man, and Cybernetics: Systems, 2016
- Deep Residual Learning for Image Recognition2016
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Gaussian Elimination and the LU Decomposition2015
- Miscible Displacement of Heavy West Sak Crude by Solvents in Slim Tube1989
- GENERALIZED INVERSE OF A MATRIX AND ITS APPLICATIONS1972