A Hybrid Regularized Multilayer Perceptron for Input Noise Immunity
- 6 January 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Artificial Intelligence
- Vol. 5 (1) , 115-126
Abstract
The immunity of multilayer perceptron (MLP) is less effective towards input noise. In this article, we have focused on the robustness of MLP with respect to input noise where noise can be additive or multiplicative. Here, we have proposed a DropConnect based regularized MLP to reduce the coadaptation among the neurons of the hidden layer. At first, we have empirically and statistically shown that by reducing coadaptation among the hidden neurons, an MLP can achieve better noise immunity. We have also empirically shown that an MLP with input noise injection and $l_{2}$ regularizer is an effective approach to improve its noise immunity. However, the results indicate that it does not adjust the coadaptation among the hidden neurons. Therefore, for further improvement, we have proposed a hybrid regularized MLP (HRMLP), where DropConnect is combined with the noise injection and $l_{2}$ regularizer. In addition to input noise, we have also verified the robustness of HRMLP with respect to $\mathbf {20\%}$ outliers in the data set. To justify the effectiveness of the proposed HRMLP, we have compared it with MLP, MLP with noise injection, MLP with $l_{2}$ regularizer, MLP with noise injection and $l_{2}$ regularizer, and MLP with DropConnect along with two state-of-the-art works based on $\mathbf {20}$ standard data sets. The experimental results for both noisy inputs and outliers confirm that the performance of HRMLP is significant compared to other methods.Keywords
This publication has 62 references indexed in Scilit:
- Regularizing the effect of input noise injection in feedforward neural networks trainingNeural Computing and Applications, 2004
- Sensitivity analysis of multilayer perceptron to input and weight perturbationsIEEE Transactions on Neural Networks, 2001
- A Quantitative Study of Fault Tolerance, Noise Immunity, and Generalization Ability of MLPsNeural Computation, 2000
- k-nearest neighbors directed noise injection in multilayer perceptron trainingIEEE Transactions on Neural Networks, 2000
- A Comparative Study of Neural Network Optimization Techniques1998
- The Effects of Adding Noise During Backpropagation Training on a Generalization PerformanceNeural Computation, 1996
- Enhanced MLP performance and fault tolerance resulting from synaptic weight noise during trainingIEEE Transactions on Neural Networks, 1994
- Sensitivity analysis of multilayer perceptron with differentiable activation functionsIEEE Transactions on Neural Networks, 1992
- Noise injection into inputs in back-propagation learningIEEE Transactions on Systems, Man, and Cybernetics, 1992
- Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networksNeural Networks, 1990