A novel method of dynamic object modeling for visual tracking is presented. The Haar transformation is first applied on the incoming image of the video to get features, which are over-complete description of the image. Then, the Fisher criteria are employed for ranking features based on their contributions to the discrimination between the tracked objects and the background. After that, the objects are modeled by the subset of top-ranked features. During tracking, a Kalman filter is used to predict the upcoming destinations of the tracked objects and the features are re-ranked by the discrimination between the objects and predicted locations. Thereafter, objects models will be updated and only discriminative features are kept in it. This proposed strategy aims to maximally maintain the basic discrimination and reduce computational cost simultaneously. To evaluate the performance of the proposed method, several experiments have been conducted on long video sequences. The experimental results show that the proposed method can handle various uncertain factors under the real world conditions and successfully track the objects in real-time.