The classification algorithm based on SVM (support vector machine) attracts more attention from researchers due to its perfect theoretical properties and good empirical results. In this paper, the properties of SV set are analyzed thoroughly, and a new learning method is introdnced to extend the SVM Classification algorithm to incremental learning area. After that, a new improved incremental SVM learning algorithm is proposed, which is based on a sifting factor. This algorithm accumulates distribution knowledge of the training sample while the incremental training is proceeded, and thus makes it possible to discard samples optimally. The theoretical analysis and experimental results show that this algorithm could not only improve the training speed, but also reduce storage cost.