微信服务号

微信订阅号

2025年7月17日 16:30 星期四
首页 > 过刊浏览>2020年第31卷第7期 >1933-1942. DOI:10.13328/j.cnki.jos.005934
PDF HTML阅读 XML下载 导出引用 引用提醒
面向细粒度草图检索的对抗训练三元组网络
作者:
作者简介:

陈健(1995-),男,学士,主要研究领域为基于内容的图像检索;郝鹏翼(1986-),女,博士,讲师,CCF专业会员,主要研究领域为机器学习,图像处理;白琮(1981-),男,博士,副教授,博士生导师,CCF专业会员,主要研究领域为计算机视觉,多媒体信息处理;陈胜勇(1973-),男,博士,教授,博士生导师,CCF杰出会员,主要研究领域为计算机视觉;马青(1982-),女,讲师,主要研究领域为图像检索.

通讯作者:

白琮,E-mail:congbai@zjut.edu.cn

基金项目:

国家重点研发计划（2018YFB1305200）；浙江省自然科学基金（LY18F020032，LY18F020034）；浙江省教育厅项目（Y201839922）


Adversarial Training Triplet Network for Fine-grained Sketch Based Image Retrieval
Author:
Fund Project:

National Key R&D Program (2018YFB1305200); Natural Science Foundation of Zhejiang Province of China (LY18F020032, LY18F020034); Zhejiang Provincial Department of Education of China (Y201839922)

  • 摘要
    • | |
  • 访问统计
    • |
  • 参考文献 [38]
    • |
  • 相似文献 [20]
    • | | |
  • 文章评论
    摘要:

    将草图作为检索示例用于图像检索称为基于草图的图像检索，简称草图检索.其中，细粒度检索问题或类内检索问题是2014年被研究者提出并快速成为广受关注的研究方向.目前研究者通常用三元组网络来解决类内检索问题，且取得了不错的效果.但是三元组网络的训练非常困难，很多情况下很难收敛甚至不收敛，且存在着容易过拟合的风险.借鉴循环生成对抗训练的思想，设计了SketchCycleGAN帮助提高三元组网络训练过程的效率，以对抗训练的方式使其参与到三元组网络的训练过程中，通过充分挖掘数据集自身信息的方式取代了利用其他数据集进行预训练的过程，在简化训练步骤的基础上取得了更好的检索性能.通过在常用的细粒度草图检索数据集上的一系列对比实验，证明了所提方法的有效性和优越性.

    Abstract:

    Sketch based image retrieval means that the sketch is used as the query in the retrieval. Fine-grained image retrieval or intra-categoryretrieval was proposed in 2014 and attracted more attentions quickly. Triplet network is often used to do fine-grained retrieval and get promising performance. However, training triplet network is quite difficult, it is hard to converge and easy to over-fit in some situations. Inspired by the adversarial training, this study proposes SketchCycleGAN to improve the efficiency of the triplet network training process. In this proposal, pre-training the networks with other database is replaced by mining the information inside the database with the help of adversarial training. That could simplify the training procedure with better performance. This proposal could get better performance than other state-of-the-art methods in a series of experiments executed on widely used databases for fine-grained sketchbased retrieval.

    参考文献
    [1] Li Y, Li W. A survey of sketch-based image retrieval. Machine Vision and Applications, 2018,29(7):1083-1100.
    [2] Kato T, Takio K, Otsu N, et al. A sketch retrieval method for full color image database-query by visual example. In:Proc. of the 11th IAPR Int'l Conf. on Pattern Recognition. 1992. 530-533.
    [3] Xu D, Alameda-pineda X, Song J, et al. Cross-paced representation learning with partial curricula for sketch-based image retrieval. IEEE Trans. on Image Processing, 2018,27(9):4410-4421.
    [4] Li B, Liang S, Sun ZX. Sketch retrieval based on topological relations. Computer Science, 2005,(12):227-231(in Chinese with English abstract).
    [5] Liang S, Sun ZX. Small sample incremental biased learning algorithm for sketch retrieval. Ruan Jian Xue Bao/journal of Software, 2009,20(5):1301-1312(in Chinese with English abstract). http://www.jos.org.cn/1000-9825/3274.htm[doi:10.3724/SP.J.1001. 2009.03274]
    [6] Bai C, Huang L, Chen JN, Pan X, Chen SY. Optimization of deep convolutional neural network for large scale image classification. Ruan Jian Xue Bao/Journal of Software, 2018,29(4):1029-1038(in Chinese with English abstract). http://www.jos.org.cn/1000-9825/5404.htm[doi:10.13328/j.cnki.jos.005404]
    [7] Fan YC, Tan XH, Zhou MQ, Zheng X. A scale invariant local descriptor for sketch based 3D model retrieva. Chinese Journal of Computers, 2017,40(11):2448-2465(in Chinese with English abstract).
    [8] Yu MY, Wu H, Guo XY, Jia Q, Guo H. Sequential feature based sketch recognition. Computer Science, 2018,45(S2):198-202(in Chinese with English abstract).
    [9] Song J, Yu Q, Song YZ, et al. Deep spatial-semantic attention for fine-grained sketch-based image retrieval. In:Proc. of the 2017 IEEE Int'l Conf. on Computer Vision (ICCV). IEEE, 2017. 5552-5561.
    [10] Li Y, Hospedales T, Song YZ, et al. Fine-grained sketch-based image retrieval by matching deformable part models. In:Proc. of the British Machine Vision Conf. British Machine Vision Association. 2014. 115.1-115.12.
    [11] Sangkloy P, Burnell N, Ham C, et al. The Sketchy database:Learning to retrieve badly drawn bunnies. ACM Trans. on Graphics (TOG), 2016,35(4):1-12.
    [12] Li Z, Tang J. Weakly supervised deep metric learning for community-contributed image retrieval. IEEE Trans. on Multimedia, 2015,17(11):1989-1999.
    [13] Li Z, Tang J, Mei T. Deep collaborative embedding for social image understanding. IEEE Trans. on Pattern analysis and Machine Intelligence, 2019,41(9):2070-2083.
    [14] Tang J, Li Z. Weakly supervised multimodal hashing for scalable social image retrieval. IEEE Trans. on Circuits and Systems for Video Technology, 2017,28(10):2730-2741.
    [15] Yu Q, Liu F, Song YZ, et al. Sketch me that shoe. In:Proc. of the 2016 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR). IEEE, 2016. 799-807.
    [16] Yu Q, Yang Y, Liu F, et al. Sketch-a-Net:A deep neural network that beats humans. Int'l Journal of Computer Vision, 2017,122(3):411-425.
    [17] Song J, Song YZ, Xiang T, et al. Deep multi-task attribute-driven ranking for fine-grained sketch-based image retrieval. BMVC, 2016, 1-11.
    [18] Huang F, Cheng Y, Jin C, et al. Deep multimodal embedding model for fine-grained sketch-based image retrieval. In:Proc. of the 40th Int'l ACM SIGIR Conf. on Research and Development in Information Retrieval-SIGIR 2017. New York:ACM Press, 2017. 929-932.
    [19] Zhang J, Shen F, Liu L, et al. Generative domain-migration hashing for sketch-to-image retrieval. In:Ferrari V, Hebert M, Sminchisescu C, et al., eds. Proc. of the ECCV 2018. Cham:Springer Int'l Publishing, 2018. 304-321.
    [20] B KP, Li D, Song J, et al. Deep factorised inverse-sketching. In:Proc. of the ECCV 2018. 2018. 37-54.
    [21] Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets. In:Advances in Neural Information Processing Systems. 2014. 2672-2680.
    [22] Zhu JY, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks. In:Proc. of the IEEE Int'l Conf. on Computer Vision. 2017. 2223-2232.
    [23] Isola P, Zhu JY, Zhou T, et al. Image-to-image translation with conditional adversarial networks. In:Proc. of the IEEE Conf. on Computer Vision and PATTERN Recognition. 2017. 1125-1134.
    [24] Hoffer E, Ailon N. Deep metric learning using triplet network. In:Proc. of the Int'l Workshop on Similarity-Based Pattern Recognition. Cham:Springer-Verlag, 2015. 84-92.
    [25] Bromley J, Guyon I, LeCun Y, et al. Signature verification using a "siamese" time delay neural network. In:Advances in Neural Information Processing Systems. 1994. 737-744.
    [26] Cheng D, Gong Y, Zhou S, et al. Person re-identification by multi-channel parts-based cnn with improved triplet loss function. In:Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. 2016. 1335-1344.
    [27] LeCun Y, Huang FJ. Loss functions for discriminative training of energy-based models. In:Proc. of the AIStats. 2005,6:34.
    [28] Abadi M, Barham P, Chen J, et al. Tensorflow:A system for large-scale machine learning. In:Proc. of the 12th {USENIX} Symp. on Operating Systems Design and Implementation ({OSDI} 16). 2016. 265-283.
    [29] Eitz M, Hays J, Alexa M. How do humans sketch objects. ACM Trans. on Graphics, 2012,31(4):44:1-44:10.
    [30] Li Y, Hospedales TM, Song YZ, et al. Free-hand sketch recognition by multi-kernel feature learning. Computer Vision and Image Understanding, 2015,137:1-11.
    [31] Wang F, Kang L, Li Y. Sketch-based 3D shape retrieval using convolutional neural networks. In:Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. 2015. 1875-1883.
    [32] Joachims T. Optimizing search engines using clickthrough data. In:Proc. of the 8th ACM SIGKDD Int'l Conf. on Knowledge Discovery and Data Mining. ACM, 2002. 133-142.
    附中文参考文献:
    [4] 李彬,梁爽,孙正兴.基于空间关系的手绘草图检索.计算机科学,2005,(12):227-231.
    [5] 梁爽,孙正兴.面向草图检索的小样本增量有偏学习算法.软件学报,2009,20(5):1301-1312. http://www.jos.org.cn/1000-9825/3274.htm[doi:10.3724/SP.J.1001.2009.03274]
    [6] 白琮,黄玲,陈佳楠,潘翔,陈胜勇.面向大规模图像分类的深度卷积神经网络优化.软件学报,2018,29(4):1029-1038. http://www.jos.org.cn/1000-9825/5404.htm[doi:10.13328/j.cnki.jos.005404]
    [7] 樊亚春,谭小慧,周明全,郑霞.基于局部多尺度的三维模型草图检索方法.计算机学报,2017,40(11):2448-2465.
    [8] 于美玉,吴昊,郭晓燕,贾棋,郭禾.基于时序特征的草图识别方法.计算机科学,2018,45(S2):198-202.
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

陈健,白琮,马青,郝鹏翼,陈胜勇.面向细粒度草图检索的对抗训练三元组网络.软件学报,2020,31(7):1933-1942

复制
分享
文章指标
  • 点击次数:3238
  • 下载次数: 5788
  • HTML阅读次数: 3787
  • 引用次数: 0
历史
  • 收稿日期:2019-05-02
  • 最后修改日期:2019-07-11
  • 在线发布日期: 2020-01-17
  • 出版日期: 2020-07-06
文章二维码
友情链接:中国科学院软件研究所中国计算机学会中国科学院国家自然科学基金委员会CCF数字图书馆Int'l J of Softw Info计算机系统应用
您是第位访问者
版权所有：中国科学院软件研究所 京ICP备05046678号-3
地址：北京市海淀区中关村南四街4号,邮政编码：100190
电话：010-62562563 传真：010-62562533 Email：jos@iscas.ac.cn
技术支持：北京勤云科技发展有限公司

京公网安备 11040202500063号