The tracking of deformable hand gesture is a very important task in vision-based HCI (human-computer interaction) research. A novel real-time tracking approach is proposed to capture the motion of deformable hand gesture with single camera. The proposed approach uses a set of 2D hand models in place of high-dimensional 3D model. It achieves auto-initialization by firstly using Bayesian classifier to do posture recognition,and then locating fingers and fingertips to fit image features to recognized posture. It solves the problem of interference among fingers during tracking successfully by the integration of K-means clustering and particle filter. Moreover,a state checking process is embedded into tracking method,and it realizes resumption from tracking failure and update of hand models automatically. Experimental results show that the proposed method can achieve continuous real-time tracking of deformable hand gesture with high precision,and thus it can meet the requirements from real-time vision-based human-computer interaction.