Feature Generation Approach with Indirect Domain Adaptation for Transductive Zero-shot Learning
Author:
Affiliation:

Clc Number:

TP181

  • Article
  • | |
  • Metrics
  • |
  • Reference [53]
  • |
  • Related [20]
  • | | |
  • Comments
    Abstract:

    In recent years, zero-shot learning has attracted extensive attention in machine learning and computer vision. The conventional inductive zero-shot learning attempts to establish the mappings between semantic and visual features for transferring the knowledge between classes. However, such approaches suffer from the projection domain shift between the seen and unseen classes. The transductive zero-shot learning is proposed to alleviate this issue by leveraging the unlabeled unseen data for domain adaptation in the training stage. Unfortunately, empirically study finds that these transductive zero-shot learning approaches, which optimize the semantic mapping and domain adaption in visual feature space simultaneously, are easy to trap in "mutual restriction", and thereby limit the potentials of both these two steps. In order to address the aforementioned issue, a novel transductive zero-shot learning approach named feature generation with indirect domain adaption (FG-IDA) is proposed, that conducts the semantic mapping and domain adaption orderly and optimizes these two steps in different spaces separately for inspiring their performance potentials and further improving the zero-shot recognition accuracy. FG-IDA is evaluated on four benchmarks, namely CUB, AWA1, AWA2, and SUN. The experimental results demonstrate the superiority of the proposed method over other transductive zero-shot learning approaches, and also show that FG-IDA achieves the state-of-the-art performances on CUB, AWA1, and AWA2 datasets. Moreover, the detailed ablation analysis is conducted and the results empirically prove the existence of the "mutual restriction" effect in direct domain adaption-based transductive zero-shot learning approaches and the effectiveness of the indirect domain adaption idea.

    Reference
    [1] Lampert CH, Nickisch H, Harmeling S. Learning to detect unseen object classes by between-class attribute transfer. In:Proc. of the CVPR. 2009. 951-958.
    [2] Akata Z, Reed S, Walter D, et al. Evaluation of output embeddings for fine-grained image classification. In:Proc. of the CVPR. 2015. 2927-2936.
    [3] Xian Y, Schiele B, Akata Z. Zero-shot learning-The good, the bad and the ugly. In:Proc. of the CVPR. 2017. 3077-3086.
    [4] Lampert CH, Nickisch H, Harmeling S. Attribute-based classification for zero-shot visual object categorization. Pattern Analysis and Machine Intelligence, 2014, 36(3):453-465.
    [5] Frome A, Corrado GS, Shlens J, et al. Devise:A deep visual-semantic embedding model. In:Proc. of the Advances in Neural Information Processing Systems. 2013. 2121-2129.
    [6] Jayaraman D, Grauman K. Zero-shot recognition with unreliable attributes. In:Proc. of the Advances in Neural Information Processing Systems. 2014. 3464-3472.
    [7] Fu Z, Xiang T, Kodirov E, et al. Zero-shot object recognition by semantic manifold distance. In:Proc. of the CVPR. 2015. 2635-2644.
    [8] Xian YQ, Sharma S, Schiele B, et al. f-VAEGAN-D2:A feature generating framework for any-shot learning. In:Proc. of the CVPR. 2019. 10275-10284.
    [9] Schonfeld E, Ebrahimi S, Sinha S, et al. Generalized zero-and few-shot learning via aligned variational autoencoders. In:Proc. of the CVPR. 2019. 8247-8255.
    [10] Fu Y, Hospedales TM, Xiang T, et al. Transductive multi-view zero-shot learning. Pattern Analysis and Machine Intelligence, 2015, 37(11):2332-2345.
    [11] Yang L, Jing LP, Yu J. Heterogeneous transductive transfer learning algorithm. Ruan Jian Xue Bao/Journal of Software, 2015, 26(11):2762-2780(in Chinese with English abstract). http://www.jos.org.cn/1000-9825/4892.html[doi:10.13328/j.cnki.jos.004892]
    [12] Ji Z, Sun T, Yu YL. Transductive discriminative dictionary learning approach for zero-shot classification. Ruan Jian Xue Bao/Journal of Software, 2017, 28(11):2961-2970(in Chinese with English abstract). http://www.jos.org.cn/1000-9825/5338.html[doi:10.13328/j.cnki.jos.005338]
    [13] Khare V, Mahajan D, Bharadhwaj H, et al. A generative framework for zero shot learning with adversarial domain adaptation. In:Proc. of the WACV. 2020. 3101-3110.
    [14] Wu J, Zhang T, Zha ZJ, et al. Self-supervised domain-aware generative network for generalized zero-shot learning. In:Proc. of the CVPR. 2020. 12767-12776.
    [15] Akata Z, Perronnin F, Harchaoui Z, et al. Label-embedding for attribute-based classification. In:Proc. of the CVPR. 2013. 819-826.
    [16] Al-Halah Z, Tapaswi M, Stiefelhagen R. Recovering the missing link:Predicting class-attribute associations for unsupervised zero-shot learning. In:Proc. of the CVPR. 2016. 5975-5984.
    [17] Norouzi M, Mikolov T, Bengio S, et al. Zero-shot learning by convex combination of semantic embeddings. arXiv:1312.5650, 2013.
    [18] Kankuekul P, Kawewong A, Tangruamsub S, et al. Online incremental attribute-based zero-shot learning. In:Proc. of the CVPR. 2012. 3657-3664.
    [19] Yang G, Liu JL, Li XR, et al. Visual feature combination approach for zero-shot learning. Ruan Jian Xue Bao/Journal of Software, 2018, 29:16-29(in Chinese with English abstract). http://www.jos.org.cn/1000-9825/18014.htm
    [20] Mikolov T, Sutskever I, Chen K, et al. Distributed representations of words and phrases and their compositionality. In:Advances in Neural Information Processing Systems. 2013. 3111-3119.
    [21] Chen L, Zhang H, Xiao J, et al. Zeroshot visual recognition using semantics-preserving adversarial embedding networks. arXiv:1712.01928, 2017.
    [22] Xian Y, Lorenz T, Schiele B, et al. Feature generating networks for zero-shot learning. In:Proc. of the CVPR. 2018. 5542-5551.
    [23] Verma VK, Rai P. A simple exponential family framework for zero-shot learning. In:Proc. of the ECML PKDD. Cham:Springer, 2017. 792-808.
    [24] Wang WL, Pu YC, Verma VK, et al. Zero-shot learning via class-conditioned deep generative models. arXiv:1711.05820, 2017.
    [25] Li Y, Swersky K, Zemel R. Generative moment matching networks. In:Proc. of the ICML. 2015. 1718-1727.
    [26] Gretton A, Borgwardt KM, Rasch MJ, et al. A kernel method for the two-sample problem. arXiv:0805.2368, 2008.
    [27] Kingma DP, Welling M. Auto-encoding variational Bayes. arXiv:1312.6114, 2013.
    [28] Goodfellow IJ, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets. In:Advances in Neural Information Processing Systems. 2014. 2672-2680.
    [29] Verma VK, Arora G, Mishra A, et al. Generalized zero-shot learning via synthesized examples. In:Proc. of the CVPR. 2018. 4281-4289.
    [30] Mishra A, Reddy SK, Mittal A, et al. A generative model for zero shot learning using conditional variational autoencoders. In:Proc. of the CVPR. 2018. 2188-2196.
    [31] Bucher M, Herbin S, Jurie F. Generating visual representations for zero-shot classification. In:Proc. of the CVPR. 2017. 2666-2673.
    [32] Lu J, Li J, Yan ZA, et al. Zero-shot learning by generating pseudo feature representations. arXiv:1703.06389, 2017.
    [33] Song J, Shen CC, Yang YZ, et al. Transductive unbiased embedding for zero-shot learning. In:Proc. of the CVPR. 2018. 1024-1033.
    [34] Kodirov E, Xiang T, Fu ZY, et al. Unsupervised domain adaptation for zero-shot learning. In:Proc. of the ICCV. 2015. 2452-2460.
    [35] Ye M, Guo YH. Zero-shot classification with discriminative semantic representation learning. In:Proc. of the CVPR. 2017. 7140-7148.
    [36] Ganin Y, Lempitsky V. Unsupervised domain adaptation by backpropagation. In:Proc. of the ICML. 2015. 1180-1189.
    [37] Zhang ZM, Saligrama V. Learning joint feature adaptation for zero-shot recognition. arXiv:1611.07593, 2016.
    [38] Mirza M, Osindero S. Conditional generative adversarial nets. arXiv:1411.1784, 2014.
    [39] Gulrajani I, Ahmed F, Arjovsky M, et al. Improved training of Wasserstein GANs. In:Proc. of the Advances in Neural Information Processing Systems. 2017. 5767-5777.
    [40] Paul A, Krishnan NC, Munjal P. Semantically aligned bias reducing zero shot learning. In:Proc. of the CVPR. 2019. 7056-7065.
    [41] Wah C, Branson S, Welinder P, et al. The Caltech-UCSD birds-200-2011 dataset. Technical Report, CNS-TR-2011-001, 2011. http://www.vision.caltech.edu/datasets/cub_200_2011/
    [42] Patterson G, Hays J. Sun attribute database:Discovering, annotating, and recognizing scene attributes. In:Proc. of the CVPR. 2012. 2751-2758.
    [43] Reed S, Akata Z, Lee H, et al. Learning deep representations of fine-grained visual descriptions. In:Proc. of the CVPR. 2016. 49-58.
    [44] Li JJ, Jin MM, Lu K, et al. Leveraging the invariant side of generative zero-shot learning. arXiv:1904.04092, 2019.
    [45] Xie GS, Liu L, Jin XB, et al. Attentive region embedding network for zero-shot learning. In:Proc. of the CVPR. 2019. 9384-9393.
    [46] Yu YL, Ji Z, Han JG, et al. Episode-based prototype generating network for zero-shot learning. In:Proc. of the CVPR. 2020. 14035-14044
    [47] Pambala AK, Dutta T, Biswas S. Generative model with semantic embedding and integrated classifier for generalized zero-shot learning. In:Proc. of the WACV. 2020. 1237-1246.
    [48] Sariyildiz MB, Cinbis RG. Gradient matching generative networks for zero-shot learning. In:Proc. of the CVPR. 2019. 2168-2178.
    [49] Li K, Min MR, Fu Y. Rethinking zero-shot learning:A conditional visual classification perspective. In:Proc. of the CVPR. 2019. 3583-3592.
    附中文参考文献:
    [11] 杨柳, 景丽萍, 于剑. 一种异构直推式迁移学习算法. 软件学报, 2015, 26(11):2762-2780. http://www.jos.org.cn/1000-9825/4892.html[doi:10.13328/j.cnki.jos.004892]
    [12] 冀中, 孙涛,于云龙. 一种基于直推判别字典学习的零样本分类方法. 软件学报, 2017, 28(11):2961-2970. http://www.jos.org. cn/1000-9825/5338.html[doi:10.13328/j.cnki.jos.005338]
    [19] 杨刚, 刘金露, 李锡荣, 许洁萍. 一种基于视觉特征组合构造的零样本学习方法. 软件学报, 2018, 29:16-29. http://www.jos. org.cn/1000-9825/18014.htm
    Cited by
    Comments
    Comments
    分享到微博
    Submit
Get Citation

黄晟,杨万里,张译,张小洪,杨丹.基于间接域适应特征生成的直推式零样本学习方法.软件学报,2022,33(11):4268-4284

Copy
Share
Article Metrics
  • Abstract:800
  • PDF: 2353
  • HTML: 1861
  • Cited by: 0
History
  • Received:December 16,2020
  • Revised:January 25,2021
  • Online: November 11,2022
  • Published: November 06,2022
You are the first2034790Visitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-4
Address:4# South Fourth Street, Zhong Guan Cun, Beijing 100190,Postal Code:100190
Phone:010-62562563 Fax:010-62562533 Email:jos@iscas.ac.cn
Technical Support:Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063