Deep Neural Network Training Method Based on Individual Differences of Training Samples
Author:
Affiliation:

Clc Number:

TP181

Fund Project:

  • Article
  • |
  • Figures
  • |
  • Metrics
  • |
  • Reference
  • |
  • Related
  • |
  • Cited by
  • |
  • Materials
  • |
  • Comments
    Abstract:

    In recent years, the performance of deep neural networks in many tasks has been comparable to or even surpassed that of humans, but its generalization ability is still far from that of humans. How to improve the generalization of the network has always been an important research direction, and a lot of fruitful research has been carried out around this direction. Many effective methods have been proposed from the perspectives of expanding and enhancing training data, suppressing model complexity through regularization, and optimizing training strategies. These methods are a global strategy for the training data set, and each sample data will be treated equally. However, due to the difference in the amount of information and noise carried by each sample data, the impact on the fitting performance and generalization performance of the model during the training process should also be different. Are some samples more likely to overfit the model during repeated iterative training? How to find these samples? Can the model obtain better generalization performance by adopting a differentiated anti-overfitting strategy for different samples? In response to these problems, a method for training deep neural networks is proposed based on individual differences in sample data. First, the pre-training model is used to evaluate each training sample to determine the fit effect of each sample to the model. Then, according to the evaluation results, the training set is divided into two subsets:samples that are easy to overfit the model and the remaining ordinary samples. Finally, two subsets of data are used to train the model. In the process, a stronger anti-overfitting strategy is adopted for the subset that is more likely to overfit the model. Through a series of experiments on various deep models on different data sets, the effect of the proposed method on typical classification tasks and fine-grained classification tasks is verified.

    Reference
    Related
    Cited by
Get Citation

李响,刘明,刘明辉,姜庆,曹扬.基于样本个体差异性的深度神经网络训练方法.软件学报,2022,33(12):4534-4544

Copy
Share
Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:August 15,2020
  • Revised:February 25,2021
  • Adopted:
  • Online: December 03,2022
  • Published: December 06,2022
You are the firstVisitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-4
Address:4# South Fourth Street, Zhong Guan Cun, Beijing 100190,Postal Code:100190
Phone:010-62562563 Fax:010-62562533 Email:jos@iscas.ac.cn
Technical Support:Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063