The background subtraction is an important method to detect the moving objects, and its difficulty is the background update. So a background reconstruction algorithm based on pixel intensity classification is presented in this paper. According to the hypothesis that the background pixel intensity appears in image sequence with maximum probability, the pixel intensity differences between sequential two frames are calculated, and the intensity values at the pixels are classified by means of these differences. For the new algorithm, neither the pre-training of the background without any moving target, nor the models of background and targets are needed. Simulation results indicate that background can be reconstructed correctly by using the new algorithm, so the target can be extracted perfectly and tracked successfully.