Time series data widely exists in daily lives, attracting more and more scholars to conduct in-depth research on it. Time series classification is an important research field of time series, and hundreds of classification algorithms have been proposed. These methods are roughly divided into distance-based methods, feature-based methods, and deep learning-based methods. The first two types of methods require manual processing of features and artificial selection of classifiers, and most deep learning-based methods are end-to-end methods and show good classification results in time series classification problems. Nevertheless, the current deep learning-based methods are rarely able to improve the network for the problem of time scale selection in time series data, and rarely integrate the network in terms of network structure to better leverage their respective advantages. In order to solve these two kinds of problems, this study proposes a multi-scale residual full convolutional neural network (MRes-FCN) structure to deal with time series problems. The structure is mainly divided into the data preprocessing stage, the stage of combining the full convolutional network and the residual network. In order to evaluate the performance of this method, this study conducted experiments on 85 public data sets of UCR and compared them with distance-based methods, feature-based methods, and deep learning-based methods. Experiments show that the proposed method has better performance than other methods, and it is better than most methods on multiple data sets.