The classification of microscopic sandstone images is a basic work in geological research, and it has an important significance in the evaluation of oil and gas reservoirs. In the automatic classification of microscopic sandstone images, due to their complex and variable micro-structures, the hand-crafted features have limited abilities to represent them. In addition, since the collection and labeling of sandstone samples are costly, labeled microscopic sandstone images are usually few. In this study, a convolutional neural network based feature representation method for small-scale data sets, called FeRNet, is proposed to effectively capture the semantic information of microscopic sandstone images and enhance their feature representation. The FeRNet has a simple structure, which reduces the quantity requirements for labeled images, and prevents the overfitting. Aiming at the problem of insufficient labeled microscopic sandstone image, the image augmentation preprocessing and a CAE network-based weight initialization strategy are proposed, to reduce the risk of overfitting. Based on the microscopic sandstone images collected from Tibet, the experiments are designed and conducted. The results show that both image augmentation and CAE network can effectively improve the training of FeRNet network, when the labeled microscopic sandstone images are few; and the FeRNet features are more capable of the representations of microscopic sandstone images than the hand-crafted features.