Sketch based image retrieval means that the sketch is used as the query in the retrieval. Fine-grained image retrieval or intra-categoryretrieval was proposed in 2014 and attracted more attentions quickly. Triplet network is often used to do fine-grained retrieval and get promising performance. However, training triplet network is quite difficult, it is hard to converge and easy to over-fit in some situations. Inspired by the adversarial training, this study proposes SketchCycleGAN to improve the efficiency of the triplet network training process. In this proposal, pre-training the networks with other database is replaced by mining the information inside the database with the help of adversarial training. That could simplify the training procedure with better performance. This proposal could get better performance than other state-of-the-art methods in a series of experiments executed on widely used databases for fine-grained sketchbased retrieval.