基于样本个体差异性的深度神经网络训练方法
作者:
作者单位:

作者简介:

李响(1982-),男,硕士,工程师,CCF会员,主要研究领域为人工智能,大数据;姜庆(1982-),男,硕士,工程师,主要研究领域为大数据,人工智能;刘明(1972-),男,博士,教授,博士生导师,CCF会员,主要研究领域为人工智能,大数据,泛在网络;曹扬(1981-),男,硕士,高级工程师,主要研究领域为大数据,人工智能;刘明辉(1990-),男,博士,主要研究领域为人工智能,大数据,对抗生成网络.

通讯作者:

李响,E-mail:lx_madcat@163.com

中图分类号:

TP181

基金项目:

贵州省科技计划(黔科合支撑[2020]4Y058)


Deep Neural Network Training Method Based on Individual Differences of Training Samples
Author:
Affiliation:

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    深度神经网络目前在许多任务中的表现已经达到甚至超越了人类的水平,但是其泛化能力和人类相比还是相去甚远.如何提高网络的泛化性,一直是重要的研究方向之一.围绕这个方向开展的大量卓有成效的研究,从扩展增强训练数据、通过正则化抑制模型复杂度、优化训练策略等角度,提出了很多行之有效的方法.这些方法对于训练数据集来说都是某种全局性质的策略,每一个样本数据都会被平等的对待.但是,每一个样本数据由于其携带的信息量、噪声等的不同,在训练过程中,对模型的拟合性能和泛化性能的影响也应该是有差异性的.针对是否一些样本在反复的迭代训练中更倾向于使得模型过度拟合,如何找到这些样本,是否可以通过对不同的样本采用差异化的抗过拟合策略使得模型获得更好的泛化性能等问题,提出了一种依据样本数据的差异性来训练深度神经网络的方法,首先使用预训练模型对每一个训练样本进行评估,判断每个样本对该模型的拟合效果;然后依据评估结果将训练集分为易使得模型过拟合的样本和普通的样本两个子集;最后,再使用两个子集的数据对模型进行交替训练,过程中对易使得模型过拟合的子集采用更强有力的抗过拟合策略.通过在不同的数据集上对多种深度模型进行的一系列实验,验证了该方法在典型的分类任务和细粒度分类任务中的效果.

    Abstract:

    In recent years, the performance of deep neural networks in many tasks has been comparable to or even surpassed that of humans, but its generalization ability is still far from that of humans. How to improve the generalization of the network has always been an important research direction, and a lot of fruitful research has been carried out around this direction. Many effective methods have been proposed from the perspectives of expanding and enhancing training data, suppressing model complexity through regularization, and optimizing training strategies. These methods are a global strategy for the training data set, and each sample data will be treated equally. However, due to the difference in the amount of information and noise carried by each sample data, the impact on the fitting performance and generalization performance of the model during the training process should also be different. Are some samples more likely to overfit the model during repeated iterative training? How to find these samples? Can the model obtain better generalization performance by adopting a differentiated anti-overfitting strategy for different samples? In response to these problems, a method for training deep neural networks is proposed based on individual differences in sample data. First, the pre-training model is used to evaluate each training sample to determine the fit effect of each sample to the model. Then, according to the evaluation results, the training set is divided into two subsets:samples that are easy to overfit the model and the remaining ordinary samples. Finally, two subsets of data are used to train the model. In the process, a stronger anti-overfitting strategy is adopted for the subset that is more likely to overfit the model. Through a series of experiments on various deep models on different data sets, the effect of the proposed method on typical classification tasks and fine-grained classification tasks is verified.

    参考文献
    相似文献
    引证文献
引用本文

李响,刘明,刘明辉,姜庆,曹扬.基于样本个体差异性的深度神经网络训练方法.软件学报,2022,33(12):4534-4544

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2020-08-15
  • 最后修改日期:2021-02-25
  • 录用日期:
  • 在线发布日期: 2022-12-03
  • 出版日期: 2022-12-06
您是第位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京市海淀区中关村南四街4号,邮政编码:100190
电话:010-62562563 传真:010-62562533 Email:jos@iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号