拉普拉斯阶梯网络
作者:
作者简介:

胡聪(1987-),男,湖北随州人,博士,讲师,主要研究领域为人工智能,模式识别,计算机视觉;吴小俊(1967-),男,博士,教授,博士生导师,CCF专业会员,主要研究领域为人工智能,模式识别,计算机视觉;舒振球(1985-),男,博士,副教授,CCF专业会员,主要研究领域为模式识别;陈素根(1982-),男,博士,副教授,主要研究领域为模式识别与智能系统.

通讯作者:

吴小俊,E-mail:xiaojun_wu_jnu@163.com

中图分类号:

TP181

基金项目:

国家自然科学基金(61373055,61672265,61603159,61702012,U1836218);教育部111引智计划(B12018);江苏省自然科学基金(BK20160293);安徽省高等学校优秀青年人才支持计划(gxyq2017026)


Laplacian Ladder Networks
Author:
Fund Project:

National Natural Science Foundation of China (61373055, 61672265, 61603159, 61702012, U1836218); 111 Project of Ministry of Education of China (B12018); Natural Science Foundation of Jiangsu Province (BK20160293); Outstanding Young Talent Support Project of Higher Educations of Anhui Province (gxyq2017026)

  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献 [33]
  • |
  • 相似文献
  • |
  • 引证文献
  • | |
  • 文章评论
    摘要:

    阶梯网络不仅是一种基于深度学习的特征提取器,而且能够应用于半监督学习中.深度学习在实现了复杂函数逼近的同时,也缓解了多层神经网络易陷入局部最小化的问题.传统的自编码、玻尔兹曼机等方法易忽略高维数据的低维流形结构信息,使用这些方法往往会获得无意义的特征表示,这些特征不能有效地嵌入到后续的预测或识别任务中.从流形学习的角度出发,提出一种基于阶梯网络的深度表示学习方法,即拉普拉斯阶梯网络LLN (Laplacian ladder network).拉普拉斯阶梯网络在训练的过程中不仅对每一编码层嵌入噪声并进行重构,而且在各重构层引入图拉普拉斯约束,将流形结构嵌入到多层特征学习中,以提高特征提取的鲁棒性和判别性.在有限的有标签数据情况下,拉普拉斯阶梯网络将监督学习损失和非监督损失融合到了统一的框架进行半监督学习.在标准手写数据数据集MNIST和物体识别数据集CIFAR-10上进行了实验,结果表明,相对于阶梯网络和其他半监督方法,拉普拉斯阶梯网络都得到了更好的分类效果,是一种有效的半监督学习算法.

    Abstract:

    Ladder networks is not only an effective deep learning-based feature extractor, but also can be applied on semi-supervised learning. Deep learning has the advantage of approximating the complicated function and alleviating the optimization difficulty associated with deep models. Autoencoders and restricted Boltzmann machines ignore the manifold information of high-dimensional data and usually achieve unmeaning features which are very difficult to use in the subsequent tasks, such as prediction and recognition. From the perspective of manifold learning, a novel deep representation method Laplacian ladder networks (LLN) is proposed, which is based on ladder networks (LN). When training LLN, LLN reconstructs noisy input and encoder layers, and adds graph Laplacian constrains to learn hierarchical representations for improving the robustness and discrimination of system. Under the condition of limited labeled data, LLN fuses the supervised learning and unsupervised learning to training in a semi-supervised manner. This study performs the experiments on the MNIST and CIFAR-10 datasets. Experimental results show that the proposed method LLN achieves superior performance compared with LN and other semi-supervised methods, and it is an effective semi-supervised method.

    参考文献
    [1] Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. In: Proc. of the Advances in Neural Information Processing Systems. 2012. 1097-1105.
    [2] Liao YY, et al. Place classification with a graph regularized deep neural network. IEEE Trans. on Cognitive and Developmental Systems, 2017,9(4):304-315.
    [3] Szegedy C, Liu W, Jia Y, et al. Going deeper with convolutions. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. 2015. 1-9.
    [4] Zhao DB, Chen YR, Lv L. Deep reinforcement learning with visual attention for vehicle classification. IEEE Trans. on Cognitive and Developmental Systems, 2017,9(4):356-367.
    [5] Graves A, Jaitly N. Towards end-to-end speech recognition with recurrent neural networks. In: Proc. of the 31st Int’l Conf. on Machine Learning (ICML 2014). 2014. 1764-1772.
    [6] Hinton G, Deng L, Yu D, et al. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Processing Magazine, 2012,29(6):82-97.
    [7] Vinyals O, Toshev A, Bengio S, et al. Show and tell: A neural image caption generator. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. 2015. 3156-3164.
    [8] Fang H, Gupta S, Iandola F, et al. From captions to visual concepts and back. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. 2015. 1473-1482.
    [9] Kiros R, Salakhutdinov R, Zemel RS. Unifying visual-semantic embeddings with multimodal neural language models. arXiv preprint arXiv:1411.2539, 2014.
    [10] Cho K, Van Merrinboer B, Gulcehre C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078, 2014.
    [11] Sutskever I, Vinyals O, Le QV. Sequence to sequence learning with neural networks. In: Proc. of the Advances in Neural Information Processing Systems. 2014. 3104-3112.
    [12] Dempster AP, Laird NM, Rubin DB. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society Series B (Methodological), 1977,39(1):1-22.
    [13] Blum A, Mitchell T. Combining labeled and unlabeled data with co-training. In: Proc. of the Int’l Conf. on Computational Learning Theory. 1998. 92-100.
    [14] Zhou ZH, Li M. Tri-training: Exploiting unlabeled data using three classifiers. IEEE Trans. on Knowledge and Data Engineering, 2005,17(11):1529-1541.
    [15] Belkin M, Niyogi P, Sindhwani V. Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. The Journal of Machine Learning Research, 2006,7(11):2399-2434.
    [16] Ding SF, Zhang N, Shi ZZ. Laplacian multi layer extreme learning machine. Ruan Jian XueBao/Journal of Software, 2017,28(10): 2599-2610(in Chinese with English abstract). http://www.jos.org.cn/1000-9825/5128.htm [doi: 10.13328/j.cnki.jos.005128]
    [17] Joachims T. Transductive inference for text classification using support vector machines. In: Proc. of the ICML, Vol.99. 1999. 200-209.
    [18] Chen SG, Wu XJ. Improved projection twin support vector machine. Acta Electronica Sinica, 2017,45(2):408-416(in Chinese with English abstract).
    [19] Hinton GE, Salakhutdinov RR. Reducing the dimensionality of data with neural networks. Science, 2006,313(5786):504-507.
    [20] Hinton GE, Osindero S, Teh YW. A fast learning algorithm for deep belief nets. Neural Computation, 2006,18(7):1527-1554.
    [21] Vincent P, Larochelle H, Lajoie I, Bengio Y, Manzagol PA. Stacked denoisingautoencoders: Learning useful representations in a deep network with a local denoising criterion. The Journal of Machine Learning Research, 2010,11(12):3371-3408.
    [22] Weston J, Ratle F, Mobahi H, et al. Deep learning via semi-supervised embedding. In: Proc. of the Neural Networks: Tricks of the Trade. Berlin, Heidelberg: Springer-Verlag, 2012. 639-655.
    [23] Goodfellow I, Courville A, Bengio Y. Large-scale feature learning with spike-and-slab sparse coding. arXiv preprint arXiv:1206. 6407, 2012.
    [24] Kingma DP, Welling M. Auto-encoding variational Bayes. arXiv preprint arXiv:1312.6114, 2013.
    [25] Zhao J, Mathieu M, Goroshin R, et al. Stacked what-where auto-encoders. arXiv preprint arXiv:1506.02351, 2015.
    [26] Rasmus A, Berglund M, Honkala M, et al. Semi-supervised learning with ladder networks. In: Advances in Neural Information Processing Systems. 2015. 3546-3554.
    [27] Yang S, Li L, Wang S, et al. A graph regularized deep neural network for unsupervised image representation learning. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. 2017. 1203-1211.
    [28] Hu C, Wu XJ. Autoencoders with drop strategy. In: Advances in Brain Inspired Cognitive Systems: Proc. of the 8th Int’l Conf. on BICS 2016. Beijing: Springer Int’l Publishing, 2016. 80-89.
    [29] LeCun Y, Bottou L, Bengio Y, et al. Gradient-based learning applied to document recognition. Proc. of the IEEE, 1998,86(11): 2278-2324.
    [30] Krizhevsky A, Hinton G. Learning multiple layers of features from tiny images. 2009. Technical Report, TR-2009, Toronto: University of Toronto, 2009. https://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf.
    附中文参考文献:
    [16] 丁世飞,张楠,史忠植.拉普拉斯多层极速学习机.软件学报,2017,28(10):2599-2610. http://www.jos.org.cn/1000-9825/5128.htm [doi: 10.13328/j.cnki.jos.005128]
    [18] 陈素根,吴小俊.改进的投影孪生支持向量机.电子学报,2017,45(2):408-416.
    相似文献
    引证文献
引用本文

胡聪,吴小俊,舒振球,陈素根.拉普拉斯阶梯网络.软件学报,2020,31(5):1525-1535

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2018-05-03
  • 最后修改日期:2018-06-16
  • 在线发布日期: 2020-05-18
  • 出版日期: 2020-05-06
文章二维码
您是第19780923位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京市海淀区中关村南四街4号,邮政编码:100190
电话:010-62562563 传真:010-62562533 Email:jos@iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号