In recent years, deep learning in the computer vision has made great progress, showing good application prospects in medical image reading. In this paper, a model with construction of two-level deep convolution neural network is designed to achieve feature extraction, feature blend, and classification of the fundus photo. By comparing with doctor's diagnosis, it is shown that the output of the model is highly consistent with the doctor's diagnosis. In addition, an improved method of fine-grained image classification using weak supervised learning is proposed. Finally, future research direction is discussed.