For speech recognition systems under noisy environment, lip-reading technique c an effectively reduce the influence of noise and improve the accurate rate o f speech recognition system by adding visual information to acoustic channel. In this paper, an effective and robust approach for lip and mouth locating and tra cking is presented to enable the information extraction under abnormal illumina tion and without special marks. This approach first locates face region with skin-color model, then finds the eyes from the face region with iterative algo rithm, modifies the position and size of face according to the position of eyes, transforms the lower part of face by specific color coordinators to clearly dis tinguish lip color from skin color, and finally describes the outline of upper l ip and lower lip with deformable template.