基于实例加权和双分类器的稳定学习算法
作者:
作者简介:

杨帅(1995-),男,博士生,主要研究领域为因果发现,领域适应;王浩(1962-),男,博士,教授,博士生导师,CCF高级会员,主要研究领域为人工智能,数据挖掘;俞奎(1979-),男,博士,教授,博士生导师,CCF专业会员,主要研究领域为机器学习,数据挖掘;曹付元(1974-),男,博士,教授,博士生导师,CCF高级会员,主要研究领域为机器学习,数据挖掘

通讯作者:

俞奎,E-mail:yukui@hfut.edu.cn

中图分类号:

TP18

基金项目:

国家重点研发计划(2020AAA0106100);国家自然科学基金(61876206);智能信息处理山西省重点实验室开放课题(CICIP2020003)


Stable Learning via Sample Reweighting and Dual Classifiers
Author:
  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献 [26]
  • |
  • 相似文献 [20]
  • | | |
  • 文章评论
    摘要:

    稳定学习的目标是利用单一的训练数据构造一个鲁棒的预测模型,使其可以对任意与训练数据具有相似分布的测试数据进行精准的分类.为了在未知分布的测试数据上实现精准预测,已有的稳定学习算法致力于去除特征与类标签之间的虚假相关关系.然而,这些算法只能削弱特征与类标签之间部分虚假相关关系并不能完全消除虚假相关关系;此外,这些算法在构建预测模型时可能导致过拟合问题.为此,提出一种基于实例加权和双分类器的稳定学习算法,所提算法通过联合优化实例权重和双分类器来学习一个鲁棒的预测模型.具体而言,所提算法从全局角度平衡混杂因子对实例进行加权来去除特征与类标签之间的虚假相关关系,从而更好地评估每个特征对分类的作用.为了完全消除数据中部分不相关特征与类标签之间的虚假相关关系以及弱化不相关特征对实例加权过程的干扰,所提算法在实例加权之前先进行特征选择筛除部分不相关特征.为了进一步提高模型的泛化能力,所提算法在训练预测模型时构建两个分类器,通过最小化两个分类器的参数差异来学习一个较优的分类界面.在合成数据集和真实数据集上的实验结果表明了所提方法的有效性.

    Abstract:

    Stable learning aims to leverage the knowledge obtained only from a single training data to learn a robust prediction model for accurately predicting label of the test data from a different but related distribution. To achieve promising performance on the test data with agnostic distributions, existing stable learning algorithms focus on eliminating the spurious correlations between the features and the class variable. However, these algorithms can only weaken part of the spurious correlations between the features and the class variable, but can not completely eliminate the spurious correlations. Furthermore, these algorithms may encounter the overfitting problem in learning the prediction model. To tackle these issues, this study proposes a sample reweighting and dual classifiers based stable learning algorithm, which jointly optimizes the weights of samples and the parameters of dual classifiers to learn a robust prediction model. Specifically, to estimate the effects of all features on classification, the proposed algorithm balances the distribution of confunders by learning global sample weights to remove the spurious correlations between the features and the class variable. In order to eliminate the spurious correlations between some irrelevant features and the class variable and weaken the influence of irrelevant features on the weighting process of samples, the proposed algorithm selects and removes some irrelevant features before sample reweighting. To further improve the generalization ability of the model, the algorithm constructs two classifiers and learns a prediction model with an optimal hyperplane by minimizing the parameter difference between the two classifiers during learning the prediction model. Using synthetic and real-world datasets, the experiments have validated the effectiveness of the proposed algorithm.

    参考文献
    [1] Fan CH, Yi JY, Tao JH, Tian ZK, Liu B, Wen ZQ. Gated recurrent fusion with joint training framework for robust end-to-end speech recognition. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021, 29: 198–209. [doi: 10.1109/TASLP.2020.3039600]
    [2] Kumar Y, Sahrawat D, Maheshwari S, Mahata D, Stent A, Yin YF, Shah RR, Zimmermann R. Harnessing gans for zero-shot learning of new classes in visual speech recognition. In: Proc. of the 34th AAAI Conf. on Artificial Intelligence. Palo Alto: AAAI Press, 2020. 2645–2652.
    [3] 王乃钰, 叶育鑫, 刘露, 凤丽洲, 包铁, 彭涛. 基于深度学习的语言模型研究进展. 软件学报, 2021, 32(4): 1082–1115. http://www.jos.org.cn/1000-9825/6169.htm
    Wang NY, Ye YX, Liu L, Feng LZ, Bao T, Peng T. Language models based on deep learning: A review. Ruan Jian Xue Bao/Journal of Software, 2021, 32(4): 1082–1115 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/6169.htm
    [4] Pei YT, Huang YP, Zou Q, Zhang XY, Wang S. Effects of image degradation and degradation removal to CNN-based image classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(4): 1239–1253. [doi: 10.1109/TPAMI.2019.2950923]
    [5] Liu CY, Li J, He L, Plaza A, Li ST, Li B. Naive gabor networks for hyperspectral image classification. IEEE Transactions on Neural Networks and Learning Systems, 2021, 32(1): 376–390. [doi: 10.1109/TNNLS.2020.2978760]
    [6] Cai Q, Pan YW, Wang Y, Liu JG, Yao T, Mei T. Learning a unified sample weighting network for object detection. In: Proc. of the 2020 IEEE/CVF Conf. on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020. 14161–14170.
    [7] Wu Y, Chen YP, Yuan L, Liu ZC, Wang LJ, Li HZ, Fu Y. Rethinking classification and localization for object detection. In: Proc. of the 2020 IEEE/CVF Conf. on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020. 10183–10192.
    [8] 祁磊, 于沛泽, 高阳. 弱监督场景下的行人重识别研究综述. 软件学报, 2020, 31(9): 2883-2902. http://www.jos.org.cn/1000-9825/6083.htm
    Qi L, Yu PZ, Gao Y. Research on weak-supervised person re-identification. Ruan Jian Xue Bao/Journal of Software, 2020, 31(9): 2883-2902 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/6083.htm
    [9] Zhuang FZ, Qi ZY, Duan KY, Xi DB, Zhu YC, Zhu HS, Xiong H, He Q. A comprehensive survey on transfer learning. Proceedings of the IEEE, 2021, 109(1): 43–76. [doi: 10.1109/JPROC.2020.3004555]
    [10] 蔡瑞初, 陈薇, 张坤, 郝志峰. 基于非时序观察数据的因果关系发现综述. 计算机学报, 2017, 40(6): 1470–1490.
    Cai RC, Chen W, Zhang K, Hao ZF. A survey on non-temporal series observational data based causal discovery. Chinese Journal of Computers, 2017, 40(6): 1470–1490 (in Chinese with English abstract).
    [11] Shen ZY, Cui P, Kuang K, Li B, Chen PX. Causally regularized learning with agnostic data selection bias. In: Proc. of the 26th ACM Int’l Conf. on Multimedia. ACM Press, 2018. 411–419.
    [12] Kuang K, Cui P, Athey S, Xiong RX, Li B. Stable prediction across unknown environments. In: Proc. of the 24th ACM SIGKDD Int’l Conf. on Knowledge Discovery & Data Mining. London: ACM Press, 2018. 1617–1626.
    [13] Kuang K, Xiong RX, Cui P, Athey S, Li B. Stable prediction with model misspecification and agnostic distribution shift. In: Proc. of the 34th AAAI Conf. on Artificial Intelligence. Palo Alto: AAAI Press, 2020. 4485–4492.
    [14] Kuang K, Li B, Cui P, Liu Y, Tao JR, Zhuang YT, Wu F. Stable prediction via leveraging seed variable. arXiv:2006.05076, 2020.
    [15] Schölkopf B, Locatello F, Bauer S, Ke NR, Kalchbrenner N, Goyal A, Bengio Y. Toward causal representation learning. Proceedings of the IEEE, 2021, 109(5): 612–634. [doi: 10.1109/JPROC.2021.3058954]
    [16] Kuang K, Zhang HT, Wu RZ, Wu F, Zhuang YT, Zhang AJ. Balance-subsampled stable prediction across unknown test data. ACM Transactions on Knowledge Discovery from Data, 2022, 16(3): 45. [doi: 10.1145/3477052]
    [17] Shen ZY, Cui P, Liu JS, Zhang T, Li B, Chen ZT. Stable learning via differentiated variable decorrelation. In: Proc. of the 26th ACM SIGKDD Int’l Conf. on Knowledge Discovery & Data Mining. ACM Press, 2020. 2185–2193.
    [18] Shen ZY, Cui P, Zhang T, Kuang K. Stable learning via sample reweighting. In: Proc. of the 34th AAAI Conf. on Artificial Intelligence. Palo Alto: AAAI Press, 2020. 5692–5699.
    [19] Zhang XX, Cui P, Xu RZ, Zhou LJ, He Y, Shen ZY. Deep stable learning for out-of-distribution generalization. In: Proc. of the 2021 IEEE/CVF Conf. on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021. 5368–5378.
    [20] Spirtes P, Glymour C, Scheines R, Heckerman D. Causation, Prediction, and Search. 2nd ed., Cambridge: MIT Press, 2000.
    [21] Peng HC, Long FH, Ding CHQ. Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(8): 1226–1238. [doi: 10.1109/TPAMI.2005.159]
    [22] Yu L, Liu H. Efficient feature selection via analysis of relevance and redundancy. The Journal of Machine Learning Research, 2004, 5: 1205–1224.
    [23] Tang C, Zhu XZ, Chen JJ, Wang PC, Liu XW, Tian J. Robust graph regularized unsupervised feature selection. Expert Systems with Applications, 2018, 96: 64–76. [doi: 10.1016/j.eswa.2017.11.053]
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

杨帅,王浩,俞奎,曹付元.基于实例加权和双分类器的稳定学习算法.软件学报,2023,34(7):3206-3225

复制
分享
文章指标
  • 点击次数:791
  • 下载次数: 2597
  • HTML阅读次数: 1454
  • 引用次数: 0
历史
  • 收稿日期:2021-07-10
  • 最后修改日期:2021-08-21
  • 在线发布日期: 2022-12-16
  • 出版日期: 2023-07-06
文章二维码
您是第20055659位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京市海淀区中关村南四街4号,邮政编码:100190
电话:010-62562563 传真:010-62562533 Email:jos@iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号