微信服务号

微信订阅号

2025年4月5日 20:42 星期六
首页 > 过刊浏览>2012年第23卷第2期 >279-288. DOI:10.3724/SP.J.1001.2012.03977
PDF HTML阅读 XML下载 导出引用 引用提醒
基于最大相关熵准则的鲁棒半监督学习算法
作者:
基金项目:

国家自然科学基金(60873054, 50909012); 国家教育部高等学校博士学科点专项科研基金(20100041120009)


Robust Semi-Supervised Learning Algorithm Based on Maximum Correntropy Criterion
Author:
  • 摘要
    • | |
  • 访问统计
    • |
  • 参考文献 [26]
    • |
  • 相似文献 [20]
    • | | |
  • 文章评论
    摘要:

    分析了噪声对半监督学习Gaussian-Laplacian 正则化(Gaussian-Laplacian regularized,简称GLR)框架的影响,针对最小二乘准则对噪声敏感的特点,结合信息论的最大相关熵准则(maximum correntropy criterion,简称MCC), 提出了一种基于最大相关熵准则的鲁棒半监督学习算法(简称GLR-MCC),并证明了算法的收敛性.半二次优化技术被用来求解相关熵目标函数.在每次迭代中,复杂的信息论优化问题被简化为标准的半监督学习问题.典型机器学习数据集上的仿真实验结果表明,在标签噪声和遮挡噪声的情况下,该算法能够有效地提高半监督学习算法性能.

    Abstract:

    This paper analyzes the problem of sensitivity to noise in the mean square criterion of Gaussian- Laplacian regularized (GLR) algorithm. A robust semi-supervised learning algorithm based on maximum correntropy criterion (MCC), called GLR-MCC, is proposed to improve the robustness of GLR along with its convergence analysis. The half quadratic optimization technique is used to simplify the correntropy optimization problem to a standard semi-supervised problem in each iteration. Experimental results on typical machine learning data sets show that the proposed GLR-MCC can effectively improve the robustness of mislabeling noise and occlusion as compared with related semi-supervised learning algorithms.

    参考文献
    [1] Wei J, Peng H. Local and global preserving based semi-supervised dimensionality reduction method. Journal of Software, 2008, 19(11):2833-2842 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/19/2833.htm [doi: 10.3724/SP.J.1001.2008. 02833]
    [2] Xiao Y, Yu J. Semi-Supervised clustering based on affinity propagation algorithm. Journal of Software, 2008,19(11):2803-2813 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/19/2803.htm [doi: 10.3724/SP.J.1001.2008.02803]
    [3] Yin XS, Hu EL, Chen SC. Discriminative semi-supervised clustering analysis with pairwise constraints. Journal of Software, 2008, 19(11):2791-2802 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/19/2791.htm [doi: 10.3724/SP.J.1001.2008. 02791]
    [4] Gao Y, Liu DY, Qi H, Liu H. Semi-Supervised K-means clustering algorithm for multi-type relational data. Journal of Software, 2008,19(11):2814-2821 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/19/2814.htm [doi: 10.3724/SP.J.1001. 2008.02814]
    [5] Chen JX, Ji DH. Graph-Based semi-supervised relation extraction. Journal of Software, 2008,19(11):2843-2852 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/19/2843.htm [doi: 10.3724/SP.J.1001.2008.02843]
    [6] Zhao Z, Liu H. Spectral feature selection for supervised and unsupervised learning. In: Ghahramani Z, ed. Proc. of the 24th Int’l Conf. on Machine Learning (ICML 2007). New York: ACM, 2007. 1151-1157. [doi: 10.1145/1273496.1273641]
    [7] Yang SH, Zha HY, Zhou SK, Hu BG. Variational graph embedding for globally and locally consistent feature extraction. In: Buntine W, ed. Proc. of the European Conf. on Machine Learning and Knowledge Discovery in Databases: Part II (ECML PKDD 2009). Berlin: Springer-Verlag, 2009. 538-553. [doi: 10.1007/978-3-642-04174-7_35]
    [8] Zhou DY, Bousquet O, Lal TN, Weston J, Sch?lkopf B. Learning with local and global consistency. Advances in Neural Information Processing Systems, 2004,16:321-328.
    [9] Li M, Zhang ZX, Huang KQ, Tan TN. Robust visual tracking based on simplified biologically inspired features. In: Bayoumi M, ed. Proc. of the 16th IEEE Int’l Conf. on Image Processing (ICIP 2009). Piscataway: IEEE Press, 2009. 4061-4064. [doi: 10.1109/ ICIP.2009.5413456]
    [10] Wang F, Zhang CS. Label propagation through linear neighborhoods. IEEE Trans. on Knowledge and Data Engineering, 2008, 20(1):55-67. [doi: 10.1109/TKDE.2007.190672]
    [11] Wang F, Zhang CS. Robust self-tuning semi-supervised learning. Neurocomputing, 2007,70(16-18):2931-2939. [doi: 10.1016/ j.neucom.2006.11.004]
    [12] He R, Hu BG, Yuan XT. Robust discriminant analysis based on nonparametric maximum entropy. In: Zhou ZH, ed. Proc. of the 1st Asian Conf. on Machine Learning: Advances in Machine Learning (ACML 2009). Berlin: Springer-Verlag, 2009. 120-134. [doi: 10.1007/978-3-642-05224-8_11]
    [13] Yuan XT, Hu BG. Robust feature extraction via information theoretic learning. In: Danyluk A, ed. Proc. of the 26th Annual Int’l Conf. on Machine Learning (ICML 2009). New York: ACM, 2009. 1193-1200. [doi: 10.1145/1553374.1553526]
    [14] Huber P. Robust Statistics. New Jersey: Wiley, 1981.
    [15] Belkin M, Matveeva I, Niyogi P. Regularization and semi-supervised learning on large graphs. Lecture Notes in Computer Science, 2004,3120:624-638. [doi: 10.1007/978-3-540-27819-1_43]
    [16] Zhu XJ, Ghahramani Z, Lafferty Z. Semi-Supervised learning using gaussian fields and harmonic functions. In: Fawcett T, Mishra N, eds. Proc. of the 20th Int’l Conf. on Machine Learning (ICML 2003). AAAI Press, 2003. 912-919.
    [17] Liu WF, Pokharel PP, Principe JC. Correntropy: Properties and applications in non-gaussian signal processing. IEEE Trans. on Signal Processing, 2007,55(11):5286-5298. [doi: 10.1109/TSP.2007.896065]
    [18] Chapelle O, Weston J, Schölkopf B. Cluster kernels for semi-supervised learning. Advances in Neural Information Processing System, 2003,15:585-592.
    [19] Belkin M, Niyogi P, Sindhwani V. Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. Journal of Machine Learning Research, 2006,7(11):2399-2434.
    [20] He R, Hu BG, Zheng WS, Guo YQ. Two-Stage sparse representation for robust recognition on large-scale database. In: Fox M, Poole D, eds. Proc. of the 24th AAAI Conf. on Artificial Intelligence (AAAI 2010). AAAI Press, 2010. 475-480.
    [21] He R, Hu BG, Yuan XT, Zheng WS. Principal component analysis based on nonparametric maximum entropy. Neurocomputing, 2010,73(10-12):1840-1852. [doi: 10.1016/j.neucom.2009.12.032]
    [22] Vapnik VN. The Nature of Statistical Learning Theory. New York: Springer-Verlag, 1995.
    [23] Tyrrell RR. Convex Analysis. New Jersey: Princeton: Princeton Press, 1970.
    [24] Nikolova M, Ng MK. Analysis of half-quadratic minimization methods for signal and image recovery. Society for Industrial and Applied Mathematics, 2005,27(3):937-966. [doi: 10.1137/030600862]
    [25] Yuan XT. Algorithm study on non-parametric kernel density clustering and feature extraction [Ph.D. Thesis]. Beijing: the Chinese Academic of Sciences, 2009 (in Chinese with English abstract).
    [26] Philips PJ, Flynn PJ, Sruggs T, Bowyer KW, Bowyer KW, Jin Chang, Hoffman K, Marques J, Jaesik Min, Worek W. Overview of the face recognition grand challenge. In: Hebert M, Kriegman D, eds. Proc. of the 2005 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition (CVPR 2005). Washington: IEEE Computer Society, 2005. 947-954. [doi: 10.1109/ CVPR.2005.268]
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

杨南海,黄明明,赫然,王秀坤.基于最大相关熵准则的鲁棒半监督学习算法.软件学报,2012,23(2):279-288

复制
分享
文章指标
  • 点击次数:5913
  • 下载次数: 9637
  • HTML阅读次数: 0
  • 引用次数: 0
历史
  • 收稿日期:2010-05-04
  • 最后修改日期:2010-10-11
  • 在线发布日期: 2012-02-07
文章二维码
友情链接:中国科学院软件研究所中国计算机学会中国科学院国家自然科学基金委员会CCF数字图书馆Int'l J of Softw Info计算机系统应用
您是第19791742位访问者
版权所有：中国科学院软件研究所 京ICP备05046678号-3
地址：北京市海淀区中关村南四街4号,邮政编码：100190
电话：010-62562563 传真：010-62562533 Email：jos@iscas.ac.cn
技术支持：北京勤云科技发展有限公司

京公网安备 11040202500063号