As a popular research direction in the field of machine learning, deep neural networks are prone to the phenomenon of unstable gradients in training, which has become an important element that restricts their development. How to avoid and control unstable gradients is an important research topic of deep neural networks. This paper analyzes the cause and effect of the unstable gradients, and reviews the main models and methods of solving the unstable gradients. Furthermore, the future research trends in the unstable gradients is discussed.