基于边界判别投影的数据降维
作者:
基金项目:

中央高校基本科研业务费专项资金(2012211020209);广东省省部产学研结合专项资金(2011B090400477);珠海市产学研合作专项资金(2011A050101005,2012D0501990016);珠海市重点实验室科技攻关项目(2012D0501990026)


Margin Discriminant Projection for Dimensionality Reduction
Author:
  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献 [35]
  • |
  • 相似文献 [20]
  • | | |
  • 文章评论
    摘要:

    为了提取具有较好判别性能的低维特征,提出了一种新的有监督的线性降维算法——边界判别投影,即,最小化同类样本间的最大距离,最大化异类样本间的最小距离,同时保持数据流形的几何形状.与经典的基于边界定义的算法相比,边界判别投影可以较好地保持数据流形的几何结构和判别结构等全局特性,可避免小样本问题,具有较低的计算复杂度,可应用于超高维的大数据降维.人脸数据集上的实验结果表明,边界判别分析是一种有效的降维算法,可应用于大数据上的特征提取.

    Abstract:

    A novel supervised linear dimensionality reduction algorithm called margin discriminant projection (MDP) is proposed to extract low-dimensional features with good performance of discriminant. MDP aims to minimize maximum distance of samples belong to the same class and maximize minimum distance of samples belong to different classes, and at the sametime preserve the geometrical structure of data manifold. Compared with classical algorithms based on the definition of margin, MDP is good at preserving the global properties, such as geometrical and discriminant structure of data manifold, and can overcome small size sample problem. Due to its low cost of computation, MDP can be directly applied on ultra-high dimensional big data dimensionality reduction. Experimental results on five face data sets show its effectiveness for feature extraction on big data.

    参考文献
    [1] Li GJ. The scientific value of big data. Research Communications of The CCF, 2012,8(9):8-15 (in Chinese with English abstract).
    [2] Tenenbaum JB, De Silva V, Langford JC. A global geometric framework for nonlinear dimensionality reduction. Science, 2000, 290(5500):2319-2323. [doi: 10.1126/science.290.5500.2319]
    [3] Roweis ST, Saul LK. Nonlinear dimensionality reduction by locally linear embedding. Science, 2000,290(5500):2323-2326. [doi: 10.1126/science.290.5500.2323]
    [4] Belkin M, Niyogi P. Laplacian eigenmaps and spectral techniques for embedding and clustering. In: Proc. of the NIPS, Vol.14. 2001. 585-591. http://machinelearning.wustl.edu/mlpapers/paper_files/nips02-AA42.pdf
    [5] Zhang Z, Zha H. Principal manifolds and nonlinear dimensionality reduction via tangent space alignment. Journal of Shanghai University (English Edition), 2004,8(4):406-424. [doi: 10.1007/s11741-004-0051-1]
    [6] Donoho DL, Grimes C. Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data. Proc. of the National Academy of Sciences, 2003,100(10):5591-5596. [doi: 10.1073/pnas.1031596100]
    [7] Weinberger KQ, Sha F, Saul LK. Learning a kernel matrix for nonlinear dimensionality reduction. In: Proc. of the 21st Int'l Conf. on Machine Learning (ICML 2004). 2004. 839-846. [doi: 10.1145/1015330.1015345]
    [8] He XF, Yan SC, Hu YX, Niyogi P, Zhang HJ. Face recognition using laplacianfaces. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2005, 27(3):328-340. [doi: 10.1109/TPAMI.2005.55]
    [9] He XF, Ma WY, Zhang HJ. Learning an image manifold for retrieval. In: Proc. of the 12th Annual ACM Int'l Conf. on Multimedia. ACM Press, 2004. 17-23. [doi: 10.1145/1027527.1027532]
    [10] Cai D, He XF. Manifold adaptive experimental design for text categorization. IEEE Trans. on Knowledge and Data Engineering, 2012,24(4):707-719. [doi: 10.1109/TKDE.2011.104]
    [11] Tang JH, Hua XS, Qi GJ, Wang M, Mei T, Wu XQ. Structure-Sensitive manifold ranking for video concept detection. In: Proc. of the ACM Conf. on Multimedia. New York: ACM Press, 2007. 852-861. [doi: 10.1145/1291233.1291430]
    [12] Hoi SCH, Lyu MR. A multimodal and multilevel ranking scheme for large-scale video retrieval. IEEE Trans. on Multimedia, 2008, 10(4):607-619. [doi: 10.1109/TMM.2008.921735]
    [13] Bengio Y, Paiement JF, Vincent P. Out-of-Sample extensions for lle, isomap, mds, eigenmaps, and spectral clustering. In: Advances in Neural Information Processing Systems. MIT Press, 2003. 177-184. http://citeseerx.ist.psu.edu/viewdoc/summary? doi=10.1.1.5.1709
    [14] Vlassis N, Motomura Y, Krose B. Supervised dimension reduction of intrinsically low dimensional data. Neural Computation, 2002, 14(1):191-215. [doi: 10.1162/089976602753284491]
    [15] Jolliffe IT. Principal Component Analysis. New York: Springer-Verlag, 1986.
    [16] Fisher RA. The statistical utilization of multiple measurements. Annals of Eugenics, 1938,8(4):376-386. [doi: 10.1111/j.1469-1809. 1938.tb02189.x]
    [17] Rao CR. The utilization of multiple measurements in problems of biological classification. Journal of the Royal Statistical Society-Series B: Statistical Methodology, 1948,10(2):159-203.
    [18] Belhumeur PN, Hespanda J, Kiregeman D. Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE Trans. on PAMI, 1997. [doi: 10.1109/34.598228]
    [19] Li HF, Jiang T, Zhang KS. Efficient and robust feature extraction by maximum margin criterion. IEEE Trans. on Neural Networks, 2006,17(1):157-165. [doi: 10.1109/TNN.2005.860852]
    [20] Zheng WM, Zou CR, Zhao L. Weighted maximum margin discriminant analysis with kernels. Neurocomputing, 2005,67:357-362. [doi: 10.1016/j.neucom.2004.12.008]
    [21] Liu QS, Tang XO, Lu HQ, Ma SD. Face recognition using kernel scatter-difference-based discriminant analysis. IEEE Trans. on Neural Networks, 2006,17(4):1081-1085. [doi: 10.1109/TNN.2006.875970]
    [22] Liu J, Chen SC, Tan XY, Zhang DQ. Comments on “efficient and robust feature extraction by maximum margin criterion”. IEEE Trans. on Neural Networks, 2007,18(6):1862-1864. [doi: 10.1109/TNN.2007.900813]
    [23] Qiu XP, Wu LD. Face recognition by stepwise nonparametric margin maximum criterion. In: Proc. of the 10th IEEE Int'l Conf. on Computer Vision (ICCV 2005), Vol.2. IEEE, 2005. 1567-1572. [doi: 10.1109/ICCV.2005.91]
    [24] Yan SC, Xu D, Zhang BY, Zhang HJ. Graph embedding: A general framework for dimensionality reduction. In: Proc. of the IEEE Computer Society Conf. on Computer Vision and Pattern Recognition (CVPR 2005), Vol.2. IEEE, 2005. 830-837. [doi: 10.1109/ CVPR.2005.170]
    [25] Yan SC, Xu D, Zhang BY, Zhang HJ, Yang Q. Graph embedding and extensions: A general framework for dimensionality reduction. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2007,29(1):40-51. [doi: 10.1109/TPAMI.2007.250598]
    [26] Weinberger KQ, Blitzer J, Saul LK. Distance metric learning for large margin nearest neighbor classification. In: Advances in Neural Information Processing Systems, Vol.18. MIT Press, 2006. 1473. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1. 117.5831
    [27] Wang F, Zhang CS. Feature extraction by maximizing the average neighborhood margin. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR 2007). IEEE, 2007. 1-8. [doi: 10.1109/CVPR.2007.383124]
    [28] Wang HX, Zheng WM, Hu ZL, Chen SB. Local and weighted maximum margin discriminant analysis. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR 2007). IEEE, 2007. 1-8. [doi: 10.1109/CVPR.2007.383039]
    [29] He XF, Cai D, Han JW. Learning a maximum margin subspace for image retrieval. IEEE Trans. on Knowledge and Data Engineering, 2008,20(2):189-201. [doi: 10.1109/TKDE.2007.190692]
    [30] Fukunaga K. Introduction to Statistical Pattern Recognition. 2nd ed., Boston: Academic Press, 1990.
    [31] Ye JP, Li Q, Xiong H, Park H, Janardan R, Kumar V. Idr/qr: An incremental dimension reduction algorithm via qr decomposition. In: Proc. of the 10th ACM SIGKDD Int'l Conf. on Knowledge Discovery and Data Mining (KDD 2004). New York, 2004. 364-373. [doi: 10.1145/1014052.1014093]
    [32] Shawe-Taylor J, Cristianini N. Kernel Methods for Pattern Analysis. Cambridge University Press, 2004.
    [33] Chen HT, Chang HW, Liu TL. Local discriminant embedding and its variants. In: Proc. of the IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, Vol.2. IEEE, 2005. 846-853. [doi: 10.1109/CVPR.2005.216]
    [34] Zhang TP, Fang B, Tang YY, Shang ZW, Xu B. Generalized discriminant analysis: A matrix exponential approach. IEEE Trans. on Systems, Man, and Cybernetics, Part B: Cybernetics, 2010,40(1):186-197. [doi: 10.1109/TSMCB.2009.2024759]
    [35] Tao Y, Yang J. Quotient vs. difference: Comparison between the two discriminant criteria. Neurocomputing, 2010,73(10): 1808-1817. [doi: 10.1016/j.neucom.2009.10.026]
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

何进荣,丁立新,李照奎,胡庆辉.基于边界判别投影的数据降维.软件学报,2014,25(4):826-838

复制
分享
文章指标
  • 点击次数:6230
  • 下载次数: 8092
  • HTML阅读次数: 3068
  • 引用次数: 0
历史
  • 收稿日期:2013-10-15
  • 最后修改日期:2014-01-27
  • 在线发布日期: 2014-03-28
文章二维码
您是第19728113位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京市海淀区中关村南四街4号,邮政编码:100190
电话:010-62562563 传真:010-62562533 Email:jos@iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号