Spectral clustering, which is one of the most representative methods in clustering analysis, receives much attention from scholars, because it does not constrain the data structure of the original samples. However, traditional spectral clustering algorithm usually contains two major limitations, i.e., it is unable to cope with the large-scale settings and complex data distribution. To overcome the above shortcomings, this study introduces a deep learning framework to improve the generalization and scalability of spectral clustering, and combines the multi-view learning to mine diverse features among data samples, finally proposes a knowledge transferring based deep consensus network for multi-view spectral clustering (CMvSC). First, considering the local invariance of single view, CMvSC adopts the local learning layer to learn the specific embedding of each view individually. Then, because of the global consistency among multiple views, CMvSC introduces the global learning layer to achieve parameter sharing and feature transferring, and learns the shared embedding in different views. Meanwhile, taking the effect of affinity matrix for spectral clustering into consideration, CMvSC learns the affinity correlation between the paired samples by training the Siamese network and designing the contrastive loss, which replaces the distance metric in traditional spectral clustering. Finally, the experimental results on four datasets demonstrate the effectiveness of the proposed CMvSC for multi-view clustering.