Journal of Multimedia, Vol 6, No 1 (2011), 39-47, Feb 2011
doi:10.4304/jmm.6.1.39-47

A New KSVM + KFD Model for Improved Classification and Face Recognition

Riadh Ksantini, Boubakeur Seddik Boufama, Imran Shafiq Ahmad

Abstract


The Kernel Support Vector Machine (KSVM) is a powerful nonlinear classification methodology where, the Support Vectors (SVs) fully describe the decision surface by incorporating local information in the Kernel space. On the other hand, the Kernel Fisher Discriminant(KFD) is a non-linear classifier which has proven to be powerful and competitive to several state-of-the-art classifiers. This paper proposes a novel KSVM + KFD model which combines these two methods. This model can be viewed as an extension to the KSVM by incorporating 'global' characteristics of the data to estimate the decision boundary in the Kernel space. On the other hand, this new model could also be considered as an improvement to the KFD by incorporating the Support Vectors (local margin concept) into the KFD formulation. The KSVM + KFD model can be reduced to the classical KSVM model so that existing KSVM softwares can be used for easy implementation. An extensive comparison of the KSVM + KFD to the KFD, KSVM, Linear Discriminant Analysis (LDA), Linear Support Vector Machine (LSVM) and the combined LSVM and LDA, performed on real data sets, has shown the advantages of our proposed model. In particular, the experiments on face recognition have clearly shown the superiority of the KSVM + KFD over other methods.


Keywords


The Kernel Fisher Discriminant Analysis, The Kernel Support Vector Machine, Nonlinear Classification, Optimization Problems, Face Recognition

References


[1] N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines and other kernel-based learning methods. Cambridge, UK: Cambridge University Press, 2000.

[2] V. N. Vapnik, Statistical Learning Theory. New York, USA: John Wiley & Sons, 1998.

[3] S. Mika et al., “Fisher Discriminant Analysis with Kernels,” In Proceedings of IEEE Neural Networks for Signal Processing Workshop, pp. 41–48, 1999.

[4] G. Baudat and B. Anouar, “Generalized Discriminant Analysis Using Kernel Approach,” Neural Computation, pp. 2385–2404, 2000.
doi:10.1162/089976600300014980
PMid:11032039

[5] G. Ratsch, T. Onoda, and K. -R. Muller, “Soft Margins for Adaboost,” Machine Learning, vol. 42, no. 3, pp. 287–320, 2000.
doi:10.1023/A:1007618119488

[6] T. Xiong and V. Cherkassky, “A Combined SVM and LDA Approach for Classification,” In Proceedings of the International Joint Conference on Neural Networks, pp. 1455–1459, 2005.
doi:10.1109/IJCNN.2005.1556089

[7] A. Jain, R. Bolle, and S. Pankanti, Eds. BIOMETRICPersonal Identification in Networked Society. London, UK: Kluwer Academic Publishers, 1999.

[8] Y. Huang, D. Xu, and T. J. Cham, “Face and Human Gait Recognition Using Image-to-Class Distance,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 20, no. 3, pp. 431–438, 2010.
doi:10.1109/TCSVT.2009.2035852

[9] D. Xu et al., “Enhancing Bilinear Subspace Learning by Element Rearrangement,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 10, pp. 1913–1920, 2009.
doi:10.1109/TPAMI.2009.51
PMid:19696459

[10] D. Xu and S. Yan, “Semi-supervised Bilinear Subspace Learning,” IEEE Transactions on Image Processing, vol. 18, no. 7, pp. 1671–1676, 2009.
doi:10.1109/TIP.2009.2018015
PMid:19447714

[11] D. Xu et al., “Convergent 2D Subspace Learning with Null Space Analysis,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 18, no. 12, pp. 1753– 1759, 2008.
doi:10.1109/TCSVT.2008.2005799

[12] D. Xu, S. Yan, and J. Luo, “Face Recognition Using Spatially Constrained Earth Mover’s Distance,” IEEE Transactions on Image Processing, vol. 17, no. 11, pp. 2256–2260, 2008.
doi:10.1109/TIP.2008.2004430
PMid:18854252

[13] D. Xu et al., “Reconstruction and Recognition of Tensor- Based Objects With Concurrent Subspaces Analysis,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 18, no. 1, pp. 36–47, 2008.
doi:10.1109/TCSVT.2007.903317

[14] T. M. Cover, “Geometrical and statistical properties of systems of linear inequalities with application in pattern recognition,” IEEE Trans. Electron. Comput., vol. 14, pp. 326–334, 1965.
doi:10.1109/PGEC.1965.264137

[15] B. Schlkopf and A. Smola, Learning With Kernels-Support Vector Machines, Regularization, Optimization and Beyond. Cambridge: MA: MIT Press, 2001.

[16] S. Mika, Kernel Fisher Discriminants. Berlin: PhD thesis, University of Technology, 2002.

[17] H.W. Kuhn and A.W. Tucker, “Nonlinear programming,” In Proceedings of the 2nd Berkeley Symposium, pp. 481– 492, 1950.

[18] R.O. Duda, P. E. Hart, and D.G. Stork, Pattern Classification (2nd Edition). Wiley-Interscience, 2000.

[19] MATLAB Bioinformatics ToolboxTM, “The MathWorksTM,” 2009. [Online]. Available: http://www.mathworks.com

[20] G.H. Golub and C.F. Van Loan, Matrix Computations (third edition). The John Hopkins University Press, 1996.

[21] T.F. Coleman and Y. Li, “A reflective newton method for minimizing a quadratic function subject to bounds on some of the variables,” SIAM Journal on Optimization, vol. 6, no. 4, pp. 1040–1058, 1996.
doi:10.1137/S1052623494240456

[22] M.L. Yiu and N. Mamoulis, “Iterative projected clustering by subspace mining,” IEEE Transactions on Knowledge and Data Engineering, vol. 17, no. 2, pp. 176–189, 2005.
doi:10.1109/TKDE.2005.29

[23] A. Asuncion and D.J. Newman, “UCI machine learning repository,” 2007, University of California, Irvine, School of Information and Computer Sciences. [Online]. Available: http://www.ics.uci.edu/_mlearn/MLRepository.html

[24] H. Yu and J. Yang, “A direct LDA algorithm for highdimensional data with application to face recognition,” Pattern Recognition, vol. 34, pp. 2067–2070, 2001.
doi:10.1016/S0031-3203(00)00162-X

[25] A.S. Georghiades, “Yale face database,” 1997, yale University, Center for Computational Vision and Control. [Online]. Available: http://cvc.yale.edu/projects/yalefaces/yalefaces.html

[26] M. Lyons et al., “Coding facial expressions with gabor wavelets,” In Proceedings of the Third IEEE International Conference on Automatic Face and Gesture Recognition, pp. 200–205, Apr 1998.
doi:10.1109/AFGR.1998.670949

[27] M.J. Lyons, J. Budynek, and S. Akamatsu, “Automatic classification of single facial images,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 21, no. 12, pp. 1357–1362, Dec 1999.
doi:10.1109/34.817413

[28] Vidit Jain and Amitabha Mukherjee, “The Indian Face Database,” 2002. [Online]. Available: http://viswww.cs.umass.edu/_vidit/IndianFaceDatabase/

[29] L. K. Chan et al., “On a t-Test for the Scale Parameter Based on Sample Percentiles,” IEEE Transactions on Reliability, vol. 22, no. 2, pp. 82–87, 2009.
doi:10.1109/TR.1973.5216039

[30] C. H. Lee, S. W. Park, W. Chang, and J. W. Park, “Improving the Performance of Multi-Class SVMs in Face Recognition with Nearest Neighbor Rule,” IEEE International Conference on Tools with Artificial Intelligence, pp. 4–11, 2003.


Full Text: PDF


Journal of Multimedia (JMM, ISSN 1796-2048)

Copyright @ 2006-2012 by ACADEMY PUBLISHER – All rights reserved.