As a hot topic in computer vision community, video saliency detection aims at continuously discovering the motion-related salient objects from the video sequences by considering the spatial and temporal information jointly. Due to the complex backgrounds, diverse motion patterns, and camera motions in video sequences, video saliency detection is a more challenging task than image saliency detection. This paper summarizes the existing methods of video saliency detection, introduces the relevant experimental datasets, and analyze the performance of some state-of-the-art methods on different datasets. First, an introduction of low-level cues based video saliency detection methods including transform analysis based method, sparse representation based method, information theory based method and visual prior based method, is presented. Then, the learning-based video saliency detection methods, which mainly include traditional methods and depth learning based methods, are discussed. Subsequently, the commonly used datasets for video saliency detection are presented, and four evaluation measures are introduced. Moreover, some state-of-the-art methods with qualitative and quantitative comparisons on different datasets are analyzed in experiments. Finally, the key issues of video saliency detection are summarized, and the future development trend is discussed.