Decomposition and Symmetric Kernel Deep Neural Network Fuzzy Support Vector Machine
Open Access
Abstract
Algorithms involving kernel functions, such as support vector machine (SVM), have attracted huge attention within the artificial learning communities. The performance of these algorithms is greatly influenced by outliers and the choice of kernel functions. This paper introduces a new version of SVM named Deep Decomposition Neural Network Fuzzy SVM (DDNN-FSVM). To this end, we consider an auto-encoder (AE) deep neural network with three layers: input, hidden, and output. Unusually, the AE’s hidden layer comprises a number of neurons greater than the dimension of the input samples, which guarantees linear data separation. The encoder operator is then introduced into the FSVM’s dual to map the training samples to high-dimension spaces. To learn the support vectors and autoencoder parameters, we introduce the loss function and regularization terms in the FSVM dual. To learn from large-scale data, we decompose the resulting model into three small-dimensional submodels using Lagrangian decomposition. To solve the resulting problems, we use SMO, ISDA, and SCG for optimization problems involving large-scale data. We demonstrate that the optimal values of the three submodels solved in parallel provide a good lower bound for the optimal value of the initial model. In addition, thanks to its use of fuzzy weights, DDNN-FSVM is resistant to outliers. Moreover, DDNN-FSVM simultaneously learns the appropriate kernel function and separation path. We tested DDNN-FSVM on several well-known digital and image datasets and compared it to well-known classifiers on the basis of accuracy, precision, f-measure, g-means, and recall. On average, DDNN-FSVM improved on the performance of the classic FSVM across all datasets and outperformed several well-known classifiers.Keywords
Funding Information
- Ministry of Science and Higher Education of the Russian Federation (075-15-2022-312)
- World-Class Research Center Program for Advanced Digital Technologies (075-15-2022-312)
This publication has 82 references indexed in Scilit:
- A Modified Hestenes and Stiefel Conjugate Gradient Algorithm for Large-Scale Nonsmooth Minimizations and Nonlinear EquationsJournal of Optimization Theory and Applications, 2015
- Structural least square twin support vector machine for classificationApplied Intelligence, 2014
- Fuzzy SVM with a New Fuzzy Membership Function to Solve the Two-Class ProblemsNeural Processing Letters, 2011
- An unsupervised approach to learn the kernel functions: from global influence to local similarityNeural Computing and Applications, 2010
- Kernel-Based Inductive Transfer2008
- Feature space perspectives for learning the kernelMachine Learning, 2007
- Learning Convex Combinations of Continuously Parameterized Basic Kernels2005
- A decision based one-against-one method for multi-class support vector machinePattern Analysis and Applications, 2004
- Gaussian Processes in Machine Learning2004
- Support-vector networksMachine Learning, 1995