数据外补偿的深度网络超分辨率重建
作者:
作者单位:

作者简介:

杨文瀚(1989-),男,湖北武汉人,学士,CCF学生会员,主要研究领域为图像增强与恶劣天气下的图像复原;夏思烽(1995-),男,本科生,主要研究领域为图像视频处理;刘家瑛(1983-),女,博士,副教授,CCF高级会员,主要研究领域为图像/视频压缩编码,增强重建与分析理解;郭宗明(1966-),男,博士,研究员,博士生导师,CCF高级会员,主要研究领域为图像处理,视频处理,多媒体通信.

通讯作者:

刘家瑛,E-mail:liujiaying@pku.edu.cn

中图分类号:

基金项目:

国家自然科学基金(61772043)


Data-Driven External Compensation Guided Deep Networks for Image Super-Resolution
Author:
Affiliation:

Fund Project:

National Natural Science Foundation of China (61772043)

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    单张图像超分辨率重建受到多对一映射的困扰.对于给定的低分辨率图像块,存在若干高分辨率图像块与之对应.基于学习的方法受此影响,学习到的逆映射规则只能预测这些高分辨率图像块的均值,从而产生视觉上模糊的超分辨率重建结果.为了弥补歧义性造成的高频细节损失,提出了一种基于深度网络、利用在线检索的数据进行高频信息补偿的图像超分辨率重建算法.该方法构建一个深度网络,通过3个分支预测高分辨率重建结果:一条旁路直接将输入的低分辨率图像输入到网络的最后一层;一条内部高频信息重建路径基于低分辨率图像回归预测高分辨率图像,重建高分辨率图像的主要结构;另一条外部高频信息补偿路径根据内部重建的结果,从在线检索到的相似图像中提取高频细节,对内部重建的结果进行细节补偿.在第2条路径中,为了有效提取高频信号并使之适应于内部重建的重建结构,在多层特征的测量和约束下,进行高频细节迁移.相比于之前基于云数据库的传统图像超分辨率方法,所提出的方法是端对端可训练的(end-to-end trainable),因此,通过在大数据上进行学习,该方法能同时建模内部重建和外部补偿,并能自动权衡两者利弊从而给出最优的重建结果.图像超分辨率重建的实验结果表明,相比于最新的超分辨率算法,所提方法在主客观评价中均取得了更加优越的性能.

    Abstract:

    Single-Image super-resolution reconstruction is undercut by the problem of ambiguity. For a given low-resolution (LR) patch, there are several corresponding high-resolution (HR) patches. Learning-Based approaches suffer from this hindrance and are only capable of learning the inverse mapping from the LR patch to the mean of these HR patches, resulting in visually blurred result. In order to alleviate the high frequency loss caused by ambiguity, this paper presents a deep network for image super-resolution utilizing the online retrieved data to compensate high-frequency details. This method constructs a deep network to predict the HR reconstruction through three paths:A bypass connection directly inputting the LR image to the last layer of the network; an internal high-frequency information inference path regressing the HR images based on the input LR image, to reconstruct the main structure of the HR images; and another external high-frequency information compensation path enhancing the results of internal inference based on the online retrieved similar images. In the second path, to effectively extract the high-frequency details adaptively for the reconstruction of the internal inference, the high-frequency details are transferred under the constraints measured by hierarchical features. Compared with previous conventional cloud-based image super-resolution methods, the proposed method is end-to-end trainable. Thus, after training on a large dataset, the proposed method is capable of modeling internal inference and external compensation, and making a good trade-off between these two terms to obtain the best reconstruction result. The experimental results on image super-resolution demonstrate the superiority of the proposed method to not only conventional data-driven image super-resolution methods but also recently proposed deep learning approaches in both subjective and objective evaluations.

    参考文献
    相似文献
    引证文献
引用本文

杨文瀚,刘家瑛,夏思烽,郭宗明.数据外补偿的深度网络超分辨率重建.软件学报,2018,29(4):900-913

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2017-04-26
  • 最后修改日期:2017-06-26
  • 录用日期:
  • 在线发布日期: 2017-11-29
  • 出版日期:
您是第位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京市海淀区中关村南四街4号,邮政编码:100190
电话:010-62562563 传真:010-62562533 Email:jos@iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号