微信服务号

微信订阅号

2025年5月10日 21:05 星期六
首页 > 过刊浏览>2018年第29卷第4期 >914-925. DOI:10.13328/j.cnki.jos.005405
PDF HTML阅读 XML下载 导出引用 引用提醒
图神经网络回归的人脸超分辨率重建
作者:
作者简介:

呼延康(1992-),男,内蒙古太仆寺旗人,硕士生,主要研究领域为人脸图像处理,3D人脸重建;余乐天(1995-),男,学士,主要研究领域为人脸图像处理,图像合成;樊鑫(1977-),男,博士,教授,博士生导师,CCF高级会员,主要研究领域为计算机视觉与图像处理,医学影像分析;罗钟铉(1966-),男,博士,教授,博士生导师,CCF专业会员,主要研究领域为计算几何,图像处理,机器视觉.

通讯作者:

樊鑫,E-mail:xin.fan@dlut.edu.cn

基金项目:

国家自然科学基金（61572096，61432003）


Graph Based Neural Network Regression Strategy for Facial Image Super-Resolution
Author:
Fund Project:

National Natural Science Foundation of China (61572096, 61432003)

  • 摘要
    • | |
  • 访问统计
    • |
  • 参考文献 [49]
    • |
  • 相似文献 [20]
    • |
  • 引证文献
    • | |
  • 文章评论
    摘要:

    人脸超分辨率（super-resolution，简称SR）即将输入模糊的低分辨率（low-resolution，简称LR）人脸图像通过一系列算法处理得到较为清晰的高分辨率（high-resolution，简称HR）人脸图像的过程.相比自然图像，不同人脸图像的相同位置通常具有相似的结构.针对人脸图像的局部结构一致性特点，提出一种新的基于图结构的人脸超分辨率神经网络回归方法.将输入低分辨率图像表示为图结构，进而为图结构中每一个节点的局部表示训练一个浅层神经网络进行超分辨率回归.与基于规则矩形网格的方法相比，图结构在描述一个像素的局部信息时，不仅考虑到图像坐标的相关性，同时也关注了纹理的相似性，能够更好地表达图像局部特征.训练过程中，利用已收敛的相邻节点的神经网络参数初始化当前节点的神经网络参数，不仅加快了神经网络的收敛速度，而且提高了预测精度.与包括深度卷积神经网络在内的基于学习的超分辨率最新算法比较，实验结果表明，所提算法取得了更高的准确率.提出的图神经网络（graph neural networks，简称GNN）并不局限于解决人脸超分辨率问题，还可以用于处理其他具有不规则拓扑结构的数据，解决不同的问题.

    Abstract:

    Facial image super-resolution (SR) generates a high-resolution (HR) facial image from a low-resolution (LR) facial image. Compared with natural images, facial images are so highly structured that local patches at similar locations across different faces share similar textures. In this paper, a novel graph-based neural network (GNN) regression is proposed to leverage this local structural information for facial image SR. Firstly, the grid representation of an input face image is converted into its corresponding graph representation, and then a shallow neural network is trained for each vertex in the graph in order to regress the SR image. Compared with its grid-based counterpart, the graph representation combines both coordinate affinity and textural similarity. Additionally, the NN weights of a vertex are initialized with those converged ones from its neighbors, resulting fast convergence for training and accurate regression. Experimental comparison with the state-of-the-art SR algorithms including those based on deep convolutional neural networks (DCNN) on two benchmark face sets demonstrate the effectiveness of the proposed method in terms of both qualitative inspections and quantitative metrics. The proposed GNN is not only able to deal with facial SR, but also has the potential to apply to data examples with any irregular topology structure.

    参考文献
    [1] Shin HC, Roth HR, Gao M, Lu L, Xu, Z, Nogues I, Summers RM. Deep convolutional neural networks for computer-aided detection:CNN architectures, dataset characteristics and transfer learning. IEEE Trans. on Medical Imaging, 2016,35(5):1285-1298.[doi:10.1109/TMI.2016.2528162]
    [2] Rastegari M, Ordonez V, Redmon J, Farhadi A. XNOR-Net:ImageNet classification using binary convolutional neural networks. In:Bastian L, Jiri M, Nicu S, Max W, eds. Proc. of the European Conf. on Computer Vision. Cham:Springer Int'l Publishing, 2016. 525-542. https://link.springer.com/chapter/10.1007/978-3-319-46493-0_32
    [3] Gatys LA, Ecker AS, Bethge M. Image style transfer using convolutional neural networks. In:Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. 2016. 2414-2423.[doi:10.1109/CVPR.2016.265]
    [4] Wang CF, Su L, Zhang WG, Huang QM. No reference video quality assessment based on 3D convolutional neural network. Ruan Jian Xue Bao/Journal of Software, 2016,27(Suppl.(2)):103-112(in Chinese with English abstract). http://www.jos.org.cn/1000-9825/16025.htm
    [5] Liu N, Li CH. Single image super-resolution reconstruction via deep convolutional neural network. China Sciencepaper, 2015, 10(2):201-206(in Chinese with English abstract).
    [6] Hu W, Cheung G, Kazui M. Graph-Based dequantization of block-compressed piecewise smooth images. IEEE Signal Processing Letters, 2016,23(2):242-246.[doi:10.1109/LSP.2015.2510379]
    [7] Pang J, Cheung G. Graph Laplacian regularization for image denoising:Analysis in the continuous domain. IEEE Trans. on Image Processing, 2017,26(4):1770-1785.[doi:10.1109/TIP.2017.2651400]
    [8] Niepert M, Ahmed M, Kutzkov K. Learning convolutional neural networks for graphs. In:Balcan MF, Weinberger KQ, eds. Proc. of the 33rd Annual Int'l Conf. on Machine Learning. New York:ACM, 2016. http://proceedings.mlr.press/v48/niepert16.html
    [9] Puy G, Kitic S, Pérez P. Unifying local and non-local signal processing with graph CNNs. arXiv:1702.07759, 2017.
    [10] Defferrard M, Bresson X, Vandergheynst P. Convolutional neural networks on graphs with fast localized spectral filtering. In:Lee DD, Sugiyama M, Luxburg UV, Guyon I, Garnett R, eds. Advances in Neural Information Processing Systems. Curran Associates, Inc., 2016. 3837-3845.
    [11] Xie S, Girshick R, Dollár P, Tu Z, He K. Aggregated residual transformations for deep neural networks. arXiv:1611.05431, 2016.
    [12] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In:Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. 2016. 770-778.[doi:10.1109/CVPR.2016.90]
    [13] Gao X, Zhang K, Tao D, Li X. Image super-resolution with sparse neighbor embedding. IEEE Trans. on Image Processing, 2012,21(7):3194-3205.[doi:10.1109/TIP.2012.2190080]
    [14] Dai S, Han M, Xu W, Wu Y, Gong Y. Soft edge smoothness prior for alpha channel super resolution. In:Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. 2007. 1-8.[doi:10.1109/CVPR.2007.383028]
    [15] Sun J, Xu Z, Shum HY. Image super-resolution using gradient profile prior. In:Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. 2008. 1-8.[doi:10.1109/CVPR.2008.4587659]
    [16] Ji H, Fermüller C. Robust wavelet-based super-resolution reconstruction:Theory and algorithm. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2009,31(4):649-660.[doi:10.1109/TPAMI.2008.103]
    [17] Belekos SP, Galatsanos NP, Katsaggelos AK. Maximum a posteriori video super-resolution using a new multichannel image prior. IEEE Trans. on Image Processing, 2010,19(6):1451-1464.[doi:10.1109/TIP.2010.2042115]
    [18] Zhu Y, Li K, Jiang J. Video super-resolution based on automatic key-frame selection and feature-guided variational optical flow. Signal Processing, Image Communication, 2014,29(8):875-886.[doi:10.1016/j.image.2014.06.005]
    [19] Li K, Zhu Y, Yang J, Jiang J. Video super-resolution using an adaptive superpixel-guided auto-regressive model. Pattern Recognition, 2016,51,59-71.[doi:10.1016/j.patcog.2015.08.008]
    [20] Su H, Zhou J, Zhang ZH. Survey of super-resolution image reconstruction methods. Acta Automatica Sinica, 2013,39(8):1202-1213(in Chinese with English abstract).
    [21] Yang J, Wright J, Huang TS, Ma Y. Image super-resolution via sparse representation. IEEE Trans. on Image Processing, 2010,19(11):2861-2873.[doi:10.1109/TIP.2010.2050625]
    [22] Zhang Y, Liu J, Yang W, Guo Z. Image super-resolution based on structure-modulated sparse representation. IEEE Trans. on Image Processing, 2015,24(9):2797-2810.[doi:10.1109/TIP.2015.2431435]
    [23] Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Shi W. Photo-Realistic single image super-resolution using a generative adversarial network. arXiv:1609.04802, 2016.[doi:10.1109/CVPR.2017.19]
    [24] Wang Z, Liu D, Yang J, Han W, Huang T. Deep networks for image super-resolution with sparse prior. In:Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. 2015. 370-378.[doi:10.1109/ICCV.2015.50]
    [25] Chakrabarti A, Rajagopalan AN, Chellappa R. Super-Resolution of face images using kernel pca-based prior. IEEE Trans. on Multimedia, 2007,9(4):888-892.[doi:10.1109/TMM.2007.893346]
    [26] Wang X, Tang X. Hallucinating face by eigentransformation. IEEE Trans. on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 2005,35(3):425-434.[doi:10.1109/TSMCC.2005.848171]
    [27] An L, Bhanu B. Face image super-resolution using 2D CCA. Signal Processing, 2014,103:184-194.[doi:10.1016/j.sigpro.2013.10. 004]
    [28] Wu W, Liu Z, He X. Learning-Based super resolution using kernel partial least squares. Image and Vision Computing, 2011,29(6):394-406.[doi:10.1016/j.imavis.2011.02.001]
    [29] Park JS, Lee SW. An example-based face hallucination method for single-frame, low-resolution facial images. IEEE Trans. on Image Processing, 2008,17(10):1806-1816.[doi:10.1109/TIP.2008.2001394]
    [30] Capel D, Zisserman A. Super-Resolution from multiple views using learnt image models. In:Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, Vol.2. 2001. Ⅱ.[doi:10.1109/CVPR.2001.991022]
    [31] Yang J, Wright J, Huang TS, Ma Y. Image super-resolution via sparse representation. IEEE Trans. on Image Processing, 2010, 19(11):2861-2873.[doi:10.1109/TIP.2010.2050625]
    [32] Li M, Cheng J, Le X, Luo HM. Super-Resolution based on sparse dictionary coding. Ruan Jian Xue Bao/Journal of Software, 2012,23(5):1315-1324(in Chinese with English abstract). http://www.jos.org.cn/1000-9825/3989.htm[doi:10.3724/SP.J.1001. 2012.03989]
    [33] Chang H, Yeung DY, Xiong Y. Super-Resolution through neighbor embedding. In:Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, Vol.1. 2004. I.[doi:10.1109/CVPR.2004.1315043]
    [34] Huang H, Wu N. Fast facial image super-resolution via local linear transformations for resource-limited applications. IEEE Trans. on Circuits and Systems for Video Technology, 2011,21(10):1363-1377.[doi:10.1109/TCSVT.2011.2163461]
    [35] Huang H, He H, Fan X, Zhang J. Super-Resolution of human face image using canonical correlation analysis. Pattern Recognition, 2010,43(7):2532-2543.[doi:10.1016/j.patcog.2010.02.007]
    [36] Gao X, Zhang K, Tao D, Li X. Joint learning for single-image super-resolution via a coupled constraint. IEEE Trans. on Image Processing, 2012,21(2):469-480.[doi:10.1109/TIP.2011.2161482]
    [37] He H, Siu WC. Single image super-resolution using Gaussian process regression. In:Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. 2011. 449-456.[doi:10.1109/CVPR.2011.5995713]
    [38] Jiang J, Chen C, Ma J, Wang Z, Wang Z, Hu R. SRLSP:A face image super-resolution algorithm using smooth regression with local structure prior. IEEE Trans. on Multimedia, 2017,19(1):27-40.[doi:10.1109/TMM.2016.2601020]
    [39] Thomaz CE, Giraldi GA. A new ranking method for principal components analysis and its application to face image analysis. Image and Vision Computing, 2010,28(6):902-913.[doi:10.1016/j.imavis.2009.11.005]
    [40] Jiang J, Hu R, Wang Z, Han Z. Noise robust face hallucination via locality-constrained representation. IEEE Trans. on Multimedia, 2014,16(5):1268-81.[doi:10.1109/TMM.2014.2311320]
    [41] Jiang J, Hu R, Wang Z, Han Z. Face super-resolution via multilayer locality-constrained iterative neighbor embedding and intermediate dictionary learning. IEEE Trans. on Image Processing, 2014,23(10):4220-4231.[doi:10.1109/TIP.2014.2347201]
    [42] Dong C, Loy CC, He K, Tang X. Image super-resolution using deep convolutional networks. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2016,38(2):295-307.[doi:10.1109/TPAMI.2015.2439281]
    [43] Huang JB, Singh A, Ahuja N. Single image super-resolution from transformed self-exemplars. In:Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. 2015. 5197-5206.[doi:10.1109/CVPR.2015.7299156]
    [44] Kim J, Kwon LJ, Mu LK. Accurate image super-resolution using very deep convolutional networks. In:Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. 2016. 1646-1654.[doi:10.1109/CVPR.2016.182]
    附中文参考文献:
    [4] 王春峰,苏荔,张维刚,黄庆明.基于3D卷积神经网络的无参考视频质量评价.软件学报,2016,27(增刊(2)):103-112. http://www.jos.org.cn/1000-9825/16025.htm
    [5] 刘娜,李翠华.基于多层卷积神经网络学习的单帧图像超分辨率重建方法.中国科技论文,2015,10(2):201-206.
    [20] 苏衡,周杰,张志浩.超分辨率图像重建方法综述.自动化学报,2013,39(8):1202-1213.
    [32] 李民,陈建,乐翔,罗环敏.稀疏字典编码的超分辨率重建.软件学报,2012,23(5):1315-1324. http://www.jos.org.cn/1000-9825/3989. htm[doi:10.3724/SP.J.1001.2012.03989]
    引证文献
引用本文

呼延康,樊鑫,余乐天,罗钟铉.图神经网络回归的人脸超分辨率重建.软件学报,2018,29(4):914-925

复制
相关视频

分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2017-04-28
  • 最后修改日期:2017-06-26
  • 在线发布日期: 2017-11-29
文章二维码
友情链接:中国科学院软件研究所中国计算机学会中国科学院国家自然科学基金委员会CCF数字图书馆Int'l J of Softw Info计算机系统应用
您是第19920949位访问者
版权所有：中国科学院软件研究所 京ICP备05046678号-3
地址：北京市海淀区中关村南四街4号,邮政编码：100190
电话：010-62562563 传真：010-62562533 Email：jos@iscas.ac.cn
技术支持：北京勤云科技发展有限公司

京公网安备 11040202500063号