Stable Learning via Sample Reweighting and Dual Classifiers
Author:
Affiliation:

Clc Number:

TP18

  • Article
  • | |
  • Metrics
  • |
  • Reference [26]
  • |
  • Related [20]
  • |
  • Cited by
  • | |
  • Comments
    Abstract:

    Stable learning aims to leverage the knowledge obtained only from a single training data to learn a robust prediction model for accurately predicting label of the test data from a different but related distribution. To achieve promising performance on the test data with agnostic distributions, existing stable learning algorithms focus on eliminating the spurious correlations between the features and the class variable. However, these algorithms can only weaken part of the spurious correlations between the features and the class variable, but can not completely eliminate the spurious correlations. Furthermore, these algorithms may encounter the overfitting problem in learning the prediction model. To tackle these issues, this study proposes a sample reweighting and dual classifiers based stable learning algorithm, which jointly optimizes the weights of samples and the parameters of dual classifiers to learn a robust prediction model. Specifically, to estimate the effects of all features on classification, the proposed algorithm balances the distribution of confunders by learning global sample weights to remove the spurious correlations between the features and the class variable. In order to eliminate the spurious correlations between some irrelevant features and the class variable and weaken the influence of irrelevant features on the weighting process of samples, the proposed algorithm selects and removes some irrelevant features before sample reweighting. To further improve the generalization ability of the model, the algorithm constructs two classifiers and learns a prediction model with an optimal hyperplane by minimizing the parameter difference between the two classifiers during learning the prediction model. Using synthetic and real-world datasets, the experiments have validated the effectiveness of the proposed algorithm.

    Reference
    [1] Fan CH, Yi JY, Tao JH, Tian ZK, Liu B, Wen ZQ. Gated recurrent fusion with joint training framework for robust end-to-end speech recognition. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021, 29: 198–209. [doi: 10.1109/TASLP.2020.3039600]
    [2] Kumar Y, Sahrawat D, Maheshwari S, Mahata D, Stent A, Yin YF, Shah RR, Zimmermann R. Harnessing gans for zero-shot learning of new classes in visual speech recognition. In: Proc. of the 34th AAAI Conf. on Artificial Intelligence. Palo Alto: AAAI Press, 2020. 2645–2652.
    [3] 王乃钰, 叶育鑫, 刘露, 凤丽洲, 包铁, 彭涛. 基于深度学习的语言模型研究进展. 软件学报, 2021, 32(4): 1082–1115. http://www.jos.org.cn/1000-9825/6169.htm
    Wang NY, Ye YX, Liu L, Feng LZ, Bao T, Peng T. Language models based on deep learning: A review. Ruan Jian Xue Bao/Journal of Software, 2021, 32(4): 1082–1115 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/6169.htm
    [4] Pei YT, Huang YP, Zou Q, Zhang XY, Wang S. Effects of image degradation and degradation removal to CNN-based image classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(4): 1239–1253. [doi: 10.1109/TPAMI.2019.2950923]
    [5] Liu CY, Li J, He L, Plaza A, Li ST, Li B. Naive gabor networks for hyperspectral image classification. IEEE Transactions on Neural Networks and Learning Systems, 2021, 32(1): 376–390. [doi: 10.1109/TNNLS.2020.2978760]
    [6] Cai Q, Pan YW, Wang Y, Liu JG, Yao T, Mei T. Learning a unified sample weighting network for object detection. In: Proc. of the 2020 IEEE/CVF Conf. on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020. 14161–14170.
    [7] Wu Y, Chen YP, Yuan L, Liu ZC, Wang LJ, Li HZ, Fu Y. Rethinking classification and localization for object detection. In: Proc. of the 2020 IEEE/CVF Conf. on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020. 10183–10192.
    [8] 祁磊, 于沛泽, 高阳. 弱监督场景下的行人重识别研究综述. 软件学报, 2020, 31(9): 2883-2902. http://www.jos.org.cn/1000-9825/6083.htm
    Qi L, Yu PZ, Gao Y. Research on weak-supervised person re-identification. Ruan Jian Xue Bao/Journal of Software, 2020, 31(9): 2883-2902 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/6083.htm
    [9] Zhuang FZ, Qi ZY, Duan KY, Xi DB, Zhu YC, Zhu HS, Xiong H, He Q. A comprehensive survey on transfer learning. Proceedings of the IEEE, 2021, 109(1): 43–76. [doi: 10.1109/JPROC.2020.3004555]
    [10] 蔡瑞初, 陈薇, 张坤, 郝志峰. 基于非时序观察数据的因果关系发现综述. 计算机学报, 2017, 40(6): 1470–1490.
    Cai RC, Chen W, Zhang K, Hao ZF. A survey on non-temporal series observational data based causal discovery. Chinese Journal of Computers, 2017, 40(6): 1470–1490 (in Chinese with English abstract).
    [11] Shen ZY, Cui P, Kuang K, Li B, Chen PX. Causally regularized learning with agnostic data selection bias. In: Proc. of the 26th ACM Int’l Conf. on Multimedia. ACM Press, 2018. 411–419.
    [12] Kuang K, Cui P, Athey S, Xiong RX, Li B. Stable prediction across unknown environments. In: Proc. of the 24th ACM SIGKDD Int’l Conf. on Knowledge Discovery & Data Mining. London: ACM Press, 2018. 1617–1626.
    [13] Kuang K, Xiong RX, Cui P, Athey S, Li B. Stable prediction with model misspecification and agnostic distribution shift. In: Proc. of the 34th AAAI Conf. on Artificial Intelligence. Palo Alto: AAAI Press, 2020. 4485–4492.
    [14] Kuang K, Li B, Cui P, Liu Y, Tao JR, Zhuang YT, Wu F. Stable prediction via leveraging seed variable. arXiv:2006.05076, 2020.
    [15] Schölkopf B, Locatello F, Bauer S, Ke NR, Kalchbrenner N, Goyal A, Bengio Y. Toward causal representation learning. Proceedings of the IEEE, 2021, 109(5): 612–634. [doi: 10.1109/JPROC.2021.3058954]
    [16] Kuang K, Zhang HT, Wu RZ, Wu F, Zhuang YT, Zhang AJ. Balance-subsampled stable prediction across unknown test data. ACM Transactions on Knowledge Discovery from Data, 2022, 16(3): 45. [doi: 10.1145/3477052]
    [17] Shen ZY, Cui P, Liu JS, Zhang T, Li B, Chen ZT. Stable learning via differentiated variable decorrelation. In: Proc. of the 26th ACM SIGKDD Int’l Conf. on Knowledge Discovery & Data Mining. ACM Press, 2020. 2185–2193.
    [18] Shen ZY, Cui P, Zhang T, Kuang K. Stable learning via sample reweighting. In: Proc. of the 34th AAAI Conf. on Artificial Intelligence. Palo Alto: AAAI Press, 2020. 5692–5699.
    [19] Zhang XX, Cui P, Xu RZ, Zhou LJ, He Y, Shen ZY. Deep stable learning for out-of-distribution generalization. In: Proc. of the 2021 IEEE/CVF Conf. on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021. 5368–5378.
    [20] Spirtes P, Glymour C, Scheines R, Heckerman D. Causation, Prediction, and Search. 2nd ed., Cambridge: MIT Press, 2000.
    [21] Peng HC, Long FH, Ding CHQ. Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(8): 1226–1238. [doi: 10.1109/TPAMI.2005.159]
    [22] Yu L, Liu H. Efficient feature selection via analysis of relevance and redundancy. The Journal of Machine Learning Research, 2004, 5: 1205–1224.
    [23] Tang C, Zhu XZ, Chen JJ, Wang PC, Liu XW, Tian J. Robust graph regularized unsupervised feature selection. Expert Systems with Applications, 2018, 96: 64–76. [doi: 10.1016/j.eswa.2017.11.053]
    Cited by
    Comments
    Comments
    分享到微博
    Submit
Get Citation

杨帅,王浩,俞奎,曹付元.基于实例加权和双分类器的稳定学习算法.软件学报,2023,34(7):3206-3225

Copy
Share
Article Metrics
  • Abstract:779
  • PDF: 2516
  • HTML: 1319
  • Cited by: 0
History
  • Received:July 10,2021
  • Revised:August 21,2021
  • Online: December 16,2022
  • Published: July 06,2023
You are the firstVisitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-4
Address:4# South Fourth Street, Zhong Guan Cun, Beijing 100190,Postal Code:100190
Phone:010-62562563 Fax:010-62562533 Email:jos@iscas.ac.cn
Technical Support:Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063