Influenced by factors like facial features, accessories, facial outer contours are extracted by the traditional geometric active contour models and conatin depressions and result in fragmentation, etc. To address these problems, according to the characteristics of human face image, the study proposes a hybrid energy based geometric active contour model via combining the energies of contour outer tension force and skin color with the global energy. First an outwards tension force, computed by neighborhoods of contour points, is added to the contour. This force makes the curve insusceptible to the facial features and accessories, but move towards to the facial outer contour. As skin color is the major feature of a human face. Skin color energy is integrated to ensure a more robust algorithm. Finally, an improved skin tone detection model is proposed based on the single Gaussian function. It could generate initial position that are close to the real facial contour, laying a good foundation for contour evolution. The proposed method gives essentially accurate face segmentations on two public face databases. Take the manually segmentations as the ground truth, the proposed method compares favorably to both traditional global and local energy algorithms. Next a more challenging set containing 100 faces of life photos with variances in pose is introduced with illumination and backgrounds. Segmentation results have validated that the proposed method could extract outer facial contour steadily and accurately under such variances.