Condensed Vector Machines: Learning Fast Machine for Large Data
Authors:
- 18 October 2010
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks
- Vol. 21 (12) , 1903-1914
Abstract
Scalability is one of the main challenges for kernel-based methods and support vector machines (SVMs). The quadratic demand in memory for storing kernel matrices makes it impossible for training on million-size data. Sophisticated decomposition algorithms have been proposed to efficiently train SVMs using only important examples, which ideally are the final support vectors (SVs). However, the ability of the decomposition method is limited to large-scale applications where the number of SVs is still too large for a computer's capacity. From another perspective, the large number of SVs slows down SVMs in the testing phase, making it impractical for many applications. In this paper, we introduce the integration of a vector combination scheme to simplify the SVM solution into an incremental working set selection for SVM training. The main objective of the integration is to maintain a minimal number of final SVs, bringing a minimum resource demand and faster training time. Consequently, the learning machines are more compact and run faster thanks to the small number of vectors included in their solution. Experimental results on large benchmark datasets shows that the proposed condensed SVMs achieve both training and testing efficiency while maintaining a generalization ability equivalent to that of normal SVMs.Keywords
This publication has 15 references indexed in Scilit:
- A dual coordinate descent method for large-scale linear SVM2008
- Generalized Core Vector MachinesIEEE Transactions on Neural Networks, 2006
- A bottom-up method for simplifying support vector solutionsIEEE Transactions on Neural Networks, 2006
- IJCNN 2001 challenge: generalization ability and text decoding2002
- A Parallel Mixture of SVMs for Very Large Scale ProblemsNeural Computation, 2002
- RSVM: Reduced Support Vector Machines2001
- 10.1162/jmlr.2003.4.6.1071Applied Physics Letters, 2000
- Input space versus feature space in kernel-based methodsIEEE Transactions on Neural Networks, 1999
- Support-vector networksMachine Learning, 1995
- A training algorithm for optimal margin classifiers1992