In this paper, a face detection algorithm for color images based on skin color and template, which is composed of skin color segmentation, template matching, and neural network verifying is presented. First, a skin color model in HSI chrominance space is used for segmenting regions in which may have faces, and then the average-face based template matching and neural network verifying methods are used for searching faces in those regions. This algorithm integrates skin color information in color images with template matching and neural network classification model in gray level images, which results in not only faster speed, but also higher robustness of the algorithm. Experimental results demonstrate the efficiency and feasibility of this algorithm.