A scheme on shot content analysis for two video retrieval applications, shot retrieval and scene structure extraction is presented. To characterize the temporal content variations in one shot, two descriptors: Dominant Color Histograms and Spatial Structure Histograms, are developed. By fusing temporal information into color content, Dominant Color Histograms for a group of frames are trying to capture the dominant colors with longer durations, which would be the colors of the focused objects or background .Spatial Structure Histograms is a set of features extracted form cilir-blob maps to describe spatial information for an individual frame.A shot with significant content changes can be segmented into several subshots subshots that are of coherent content,and shot similarity measure can be computed from the similarity between corresponding subshots.Scene structure is extracted by analyzing the competition of splitting and merging forces.Experimental resl-world sports video show that the proposed approaches can achieve the best performance on shot retrievals and promising results on scene stucture extraction.