图像风格转换技术已经融入到人们的生活中,并被广泛应用于图像艺术化、卡通化、图像着色、滤镜处理和去遮挡等实际场景中,因此,图像风格转换具有重要的研究意义与应用价值.StarGAN是近年来用于多域图像风格转换的生成对抗网络框架.StarGAN通过简单地下采样提取特征,然后通过上采样生成图片,但是生成图片的背景颜色信息、人物脸部的细节特征会与输入图像有较大差异.对StarGAN的网络结构进行改进,通过引入U-Net和边缘损失函数,提出了用于图像风格转换的UE-StarGAN模型.同时,将类别编码器引入到UE-StarGAN模型的生成器中,构建了融合类别编码器的小样本图像风格转换模型,实现了小样本的图像风格转换.实验结果表明:该模型可以提取到更精细的特征,在小样本的情况下具有一定的优势,以此进行图像风格转换后的图片无论是定性分析还是定量分析都有一定的提升,验证了所提模型的有效性.

The image style transferring technology has been widely integrated into people’s life, and it is widely used in image artistry, cartoon, picture coloring, filter processing, and occlusion removal of the practical scenarios, so image style transfering has an important research significance and application value. StarGAN is a generative adversarial network framework for multi-domain image style transfering in recent years. StarGAN extracts features through simple down-sampling, and then generates images through up-sampling. Nevertheless, the background color information and detailed features of people’s faces in the generated images are quite different from those in the input images. In this study, by improving the network structure of StarGAN, after analyzing the existing problems of the StarGAN, a UE-StarGAN model for image style transfering is proposed by introducing U-Net and edge-promoting adversarial loss function. At the same time, the class encoder is introduced into the generator of UE-StarGAN, and a small sample image style transfering model is designed to realize the small sample image style transfer. The results of this experiment show that the model can extract more detailed features, have some advantages in the case of small sample size, and to a certain extent, the qualitative and quantitative analysis results of the images can be improved after the image style transfering, which verifies the effectiveness of the proposed model.

国家自然科学基金(61976217,61976216)