A novel method to segment image is proposed considering the intensity roughness and the color purity. HIS color space is used to express pixel color information. Not only the pixel properties but also the pixel group properties are considered. An image is divided into blocks. For each block, the mean and variance values can be seen as the pixel group properties. Because the hue value will be very sensitive when the intensity and saturation values are small, the color purity is used as the weight on the hue value. The image segmentation is done with the approach of the combination of region growing based on block and vector quantization. The experiments confirm this method is suitable for many kinds of images.