This paper presents a new framework for spatiotemporal alignment of two video sequences. It proposes Intra-video and inter-video matching strategy for spatial alignment; modifies Dynamic Time Warping for temporal alignment. Intra-video matching tracks feature points and binds them together. Contextual inter-video matching uses track correspondences to provide initial feature correspondences for inter-video frame matching and updates track correspondences using frame-matching results. The proposed matching strategy makes best use of coherency of source videos and improves coherency of aligned video, stability and efficiency of alignment. The Modified Dynamic Time Warping establishes frame correspondences by minimizing global differences between them, keeps temporal order of frames, and handles nonlinear misalignment of videos. The proposed method can successfully align videos viewing different events recorded by independently moving cameras. Experimental results and comparison show that great improvements on stability and efficiency of video matching together with coherency of aligned video are reached.