Differential cryptanalysis is an important method in the field of block cipher. The key point of differential cryptanalysis is to find a differential distinguisher with longer rounds or higher probability. Firstly, the method of generating data set is described which is used to train a differential distinguisher based on deep learning. At the same time, the differential distinguisher of two kinds of lightweight block cipher is trained, SIMON32 and SPECK32, based on convolutional neural networks (CNN) and residual neural network (ResNet). In addition, two differential distinguishers are compared and it is found that ResNet is good at differential distinguisher of SIMON32, CNN is good at SPECK32 when considering time and accuracy. Next, the influence of the number of convolution operations of the network model is studied on the accuracy of the neural distinguisher, and it is found that adding the number of convolution layers of the CNN and the number of residual blocks of the ResNet model will cause the accuracy decrease compared with original networks. Finally, some suggestions are given to select networks and parameters when constructing a differential distinguisher based on deep learning, i.e., the CNN with low convolutional layers and the ResNet with low residual blocks should be considered as the first choose.