Efficient Multi-Class Probabilistic SVMs on GPUs
- 19 August 2018
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Knowledge and Data Engineering
- Vol. 31 (9) , 1693-1706
Abstract
Recently, many researchers have been working on improving other traditional machine learning algorithms (besides deep learning) using high-performance hardware such as Graphics Processing Units (GPUs). The recent success of machine learning is not only due to more effective algorithms, but also more efficient systems and implementations. In this paper, we propose a novel and efficient solution to multi-class SVMs with probabilistic output (MP-SVMs) accelerated by GPUs. MP-SVMs are an important technique for many pattern recognition applications. However, MP-SVMs are very time-consuming to use, because using an MP-SVM classifier requires training many binary SVMs and performing probability estimation by combining results of all the binary SVMs. GPUs have much higher computation capability than CPUs and are potentially excellent hardware to accelerate MP-SVMs. Still, two key challenges for efficient GPU accelerations for MP-SVM are: (i) many kernel values are repeatedly computed as a binary SVM classifier is trained iteratively, resulting in repeated accesses to the high latency GPU memory; (ii) performing training or estimating probability in a highly parallel way requires a much larger memory footprint than the GPU memory. To overcome the challenges, we propose a solution called GMP-SVM which exploits two-level (i.e., binary SVM level and MP-SVM level) optimization for training MP-SVMs and high parallelism for estimating probability. GMP-SVM reduces high latency memory accesses and memory consumption through batch processing, kernel value reusing and sharing, and support vector sharing. Experimental results show that GMP-SVM outperforms the GPU baseline by two to five times, and LibSVM with OpenMP by an order of magnitude. Also, GMP-SVM produces the same SVM classifier as LibSVM.Keywords
Funding Information
- MOE AcRF Tier 1 (T1 251RES1610)
- Tier 2 (MOE2017-T2-1-122)
- Guangdong Special Branch Plans Young Talent with Scientific and Technological Innovation (2016TQ03X445)
- Guangzhou Municipal Science and Technology Program key projects (2019-03-01-06-3002-0003)
- Guangzhou Tianhe District Science and Technology Planning (201702YH112)
This publication has 25 references indexed in Scilit:
- Decision tree construction on GPU: ubiquitous parallel computing approachComputing, 2013
- CloudSVM: Training an SVM Classifier in Cloud Computing SystemsLecture Notes in Computer Science, 2013
- LIBSVMACM Transactions on Intelligent Systems and Technology, 2011
- Pegasos: primal estimated sub-gradient solver for SVMMathematical Programming, 2010
- The WEKA data mining softwareACM SIGKDD Explorations Newsletter, 2009
- Medical image retrieval with probabilistic multi-class support vector machine classifiers and adaptive similarity fusionComputerized Medical Imaging and Graphics, 2008
- A note on Platt’s probabilistic outputs for support vector machinesMachine Learning, 2007
- Parallel Sequential Minimal Optimization for the Training of Support Vector MachinesIEEE Transactions on Neural Networks, 2006
- A comparison of methods for multiclass support vector machinesIEEE Transactions on Neural Networks, 2002
- Support vector machinesIEEE Intelligent Systems and their Applications, 1998