基于自适应权重的多源部分域适应
作者:
通讯作者:

田青,E-mail:tianqing@nuist.edu.cn

基金项目:

国家自然科学基金(62176128); 计算机软件新技术国家重点实验室开放课题(KFKT2022B06); 中央高校基本科研业务费(NJ2022028); 江苏省“青蓝工程”优秀青年骨干教师人才计划; 江苏省研究生科研实践创新计划(KYCX22_1205)


Adaptive Weight-induced Multi-source Partial Domain Adaptation
Author:
  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献 [36]
  • |
  • 相似文献 [20]
  • | | |
  • 文章评论
    摘要:

    作为机器学习的一个新兴领域, 多源部分域适应(MSPDA)问题由于其源域自身的复杂性、领域之间的差异性以及目标域自身的无监督性, 给相关研究带来了挑战, 以致目前鲜有相关工作被提出. 在该场景下, 多个源域中的无关类样本在域适应过程中会造成较大的累积误差和负迁移. 此外, 现有多源域适应方法大多未考虑不同源域对目标域任务的贡献度不同. 因此, 提出基于自适应权重的多源部分域适应方法(AW­MSPDA).首先, 构建了多样性特征提取器以有效利用源域的先验知识; 同时, 设计了多层次分布对齐策略从不同层面消除了分布差异,促进了正迁移; 此外, 为量化不同源域贡献度以及过滤源域无关类样本, 利用相似性度量以及伪标签加权方式构建自适应权重; 最后, 通过大量实验验证了所提出AW­MSPDA算法的泛化性以及优越性.

    Abstract:

    As an emerging field of machine learning, multi-source partial domain adaptation (MSPDA) poses challenges to related research due to the complexities of the involved source domains, the diversities between the domains, and the unsupervised nature of the target domain itself, leading to rarely few works presented. In this scenario, the irrelevant class samples in multiple-source domains will cause large cumulative errors and negative transfer during domain adaptation. In addition, most of the existing multi­source domain adaptation methods do not consider the different contributions of different source domains to the target domain tasks. Therefore, this study proposes an adaptive weight­induced multi­source partial domain adaptation (AW-MSPDA). Firstly, a diverse feature extractor is constructed to effectively utilize the prior knowledge of the source domain. Meanwhile, multi­level distribution alignment strategy is constructed to eliminate distribution discrepancies from different levels to promote positive transfer. Moreover, the pseudo­label weighting and similarity measurement are used to construct adaptive weights to quantify the contribution of different source domains and filter samples which are irrelevant to the source domain. Finally, the generalization and performance superiority of the proposed AW­MSPDA algorithm are evaluated by extensive experiments.

    参考文献
    [1] Ganin Y, Ustinova E, Ajakan H, et al. Domain-adversarial training of neural networks. Machine Learning Research, 2016, 17(1):2096-2030.
    [2] Long MS, Cao Y, Wang JM, et al. Learning transferable features with deep adaptation networks. In:Proc. of the 32nd Int'l Conf. on Machine Learning. arXiv:1502.02791, 2015.
    [3] Tzeng E, Hoffman J, Saenko K, et al. Adversarial discriminative domain adaptation. In:Proc. of the 2017 IEEE/CVF Conf. on Computer Vision and Pattern Recognition. Hawaii:IEEE, 2017. 7167-7176.
    [4] Xu RJ, Chen ZL, Zuo WM, et al. Deep cocktail network:Multi-source unsupervised domain adaptation with category shift. In:Proc. of the 2018 IEEE/CVF Conf. on Computer Vision and Pattern Recognition. Salt Lake City:IEEE, 2018. 3964-3973.
    [5] Peng XC, Bai QX, Xia X, et al. Moment matching for multi-source domain adaptation. In:Proc. of the 17th Int'l Conf. on Computer Vision. 2019. 1406-1415.
    [6] Zhao SC, Wang GZ, Zhang SH, et al. Multi-source distilling domain adaptation. In:Proc. of the 2020 AAAI Conf. on Artificial Intelligence. 2020. 12975-12983.
    [7] Fu YY, Zhang M, Xu X, et al. Partial feature selection and alignment for multi-source domain adaptation. In:Proc. of the 2021 IEEE/CVF Conf. on Computer Vision and Pattern Recognition. IEEE, 2021. 16654-16663.
    [8] Ni J, Qiu Q, Chellappa R. Subspace interpolation via dictionary learning for unsupervised domain adaptation. In:Proc. of the 2013 IEEE/CVF Conf. on Computer Vision and Pattern Recognition. Portland:IEEE, 2013. 692-699.
    [9] Yosinski J, Clune J, Bengio Y, et al. How transferable are features in deep neural networks. In:Advances in Neural Information Processing Systems 27. 2014. 3320-3328.
    [10] Blitzer J, Crammer K, Kulesza A, et al. Learning bounds for domain adaptation. In:Advances in Neural Information Processing Systems 20. 2007. 129-136.
    [11] Hoffman J, Mohri M, Zhang NS. Algorithms and theory for multiple-source adaptation. In:Advances in Neural Information Processing Systems 31. 2018. 8256-8266.
    [12] Li YT, Murias M, Major S, et al. Extracting relationships by multi-domain matching. In:Advances in Neural Information Processing Systems 31. 2018. 6799-6810.
    [13] Zhao H, Zhang SH, Wu GH, et al. Adversarial multiple source domain adaptation. In:Advances in Neural Information Processing System 31. 2018. 8568-8579.
    [14] Zhu Y, Zhuang F, Wang D. Aligning domain-specific distribution and classifier for cross-domain classification from multiple sources. In:Proc. of the 2019 AAAI Conf. on Artificial Intelligence. 2019. 5989-5996.
    [15] Zhang J, Zhou WE, Chen XQ, et al. Multisource selective transfer framework in multiobjective optimization problems. IEEE Trans. on Evolutionary Computation, 2019, 24(3):424-438.
    [16] Zhao SC, Li B, Xu PF, et al. Madan:Multi-source adversarial domain aggregation network for domain adaptation. Int'l Journal of Computer Vision, 2021, 129(8):2399-2424.
    [17] Duan LX, Tsang IW, Xu D. Domain transfer multiple kernel learning. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2012, 34(3):465-479.
    [18] Gretton A, Borgwardt KM, Rasch MJ, et al. A kernel two-sample test. Machine Learning Research, 2012, 13(1):723-773.
    [19] Dorri F, Ghodsi A. Adapting component analysis. In:Proc. of the 12th IEEE Int 'l Conf. on Data Mining. Brussels:IEEE, 2012. 846-851.
    [20] Zhang K, Schölkopf B, Muandet K, et al. Domain adaptation under target and conditional shift. In:Proc. of the 30th Int'l Conf. on Machine Learning. 2013. 819-827.
    [21] Gong MM, Zhang K, Liu TL, et al. Domain adaptation with conditional transferable components. In:Proc. of the 33rd Int'l Conf. on Machine Learning. 2016. 2839-2848.
    [22] Cao K, Tu ZP, Ming Y. Class conditional distribution alignment for domain adaptation. Control Theory and Technology, 2020, 18(1):72-80.
    [23] Pan SJ, Tsang IW, Kwok JT, et al. Domain adaptation via transfer component analysis. IEEE Trans. on Neural Networks, 2011, 22(2):199-210.
    [24] Tzeng E, Hoffman J, Zhang N, et al. Deep domain confusion:Maximizing for domain invariance. arXiv:1412.3474, 2014.
    [25] Long MS, Zhu H, Wang JM, et al. Deep transfer learning with joint adaptation networks. In:Proc. of the 34th Int'l Conf. on Machine Learning. arXiv:1605.06636, 2017.
    [26] Yu CH, Wang JD, Chen YQ, et al. Transfer learning with dynamic adversarial adaptation network. In:Proc. of the 2019 IEEE Int'l Conf. on Data Mining. Beijing:IEEE, 2019. 778-786.
    [27] Ganin Y, Lempitsky V. Unsupervised domain adaptation by backpropagation. In:Proc.of the 32nd Int'l Conf. on Machine Learning. arXiv:1409.7495, 2015.
    [28] Saenko K, Kulis B, Fritz M, et al. Adapting visual category models to new domains. In:Proc. of the 11th European Conf. on Computer Vision. Heraklion:Springer, 2010. 213-226.
    [29] Venkateswara H, Eusebio J, Chakraborty S, et al. Deep hashing network for unsupervised domain adaptation. In:Proc. of the 2017 IEEE Conf. on Computer Vision and Pattern Recognition. Honolulu:IEEE, 2017. 5385-5394.
    [30] Sun B, Feng J, Saenko K. Return of frustratingly easy domain adaptation. In:Proc. of the 2016 AAAI Conf. on Artificial Intelligence. arXiv:1511.05547, 2015.
    [31] Saito K, Watanabe K, Ushiku Y, et al. Maximum classifier discrepancy for unsupervised domain adaptation. In:Proc. of the 2018 IEEE/CVF Conf. on Computer Vision and Pattern Recognition. Salt Lake City:IEEE, 2018. 3723-3732.
    [32] Wang H, Xu MH, Ni BB, et al. Learning to combine:Knowledge aggregation for multi-source domain adaptation. In:Proc. of the 16th European Conf. on Computer Vision. Glasgow:Springer, 2020. 727-744.
    [33] Wang SS, Wang B, Zhang Z, et al. Class-aware sample reweighting optimal transport for multi-source domain adaptation. Neurocomputing, 2023, 523:213-223.
    [34] Cao ZJ, Ma LJ, Long MS, et al. Partial adversarial domain adaptation. In:Proc. of the 15th European Conf. on Computer Vision. Munich:Springer, 2018. 139-155.
    [35] Cao ZJ, Long MS, Wang JM, et al. Partial transfer learning with selective adversarial networks. In:Proc. of the 2018 IEEE/CVF Conf. on Computer Vision and Pattern Recognition. Salt Lake City:IEEE, 2018. 2724-2732.
    [36] Cao ZJ, You KC, Long MS, et al. Learning to transfer examples for partial domain adaptation. In:Proc. of the 2019 IEEE/CVF Conf. on Computer Vision and Pattern Recognition. Long Beach:IEEE, 2019. 2980-2989.
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

田青,孙灿宇,储奕.基于自适应权重的多源部分域适应.软件学报,2024,35(4):1703-1716

复制
分享
文章指标
  • 点击次数:720
  • 下载次数: 3489
  • HTML阅读次数: 1316
  • 引用次数: 0
历史
  • 收稿日期:2023-05-15
  • 最后修改日期:2023-07-07
  • 在线发布日期: 2023-09-11
  • 出版日期: 2024-04-06
文章二维码
您是第位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京市海淀区中关村南四街4号,邮政编码:100190
电话:010-62562563 传真:010-62562533 Email:jos@iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号