An algorithm based on disparity points to solve the occlusion problem in the process of building high-quality stereo disparity map is presented in this paper. It is firstly proved that the disparity curve corresponding to a pair of epipolar-line images may be approximated by a group of piece-wise straight lines, and then the definition of disparity point is introduced. In the parameterization of a disparity point, two parameters are used to describe left and right occlusions so that the occlusion problem can be successfully solved in the approach. By analyzing intensity property of a disparity point and its neighbor points, an approach which combines stepwise hypothesis-verification strategy and Marquardt-Levenberg (M-L) algorithm is devised to extract the candidate disparity points from the epipolar images, and then aperiodic dynamic programming is employed to search the epipolar-optimal disparity function. The proposed method is tested by using the international standard image data and compared with other methods, and the experimental results show that its performance is the best among epipolar-optimal methods and worse than some excellent global-optimal approaches, but its complexity is much lower than the global-optimal approaches.