Image intelligent colorization using generative adversarial networks

Abstract

The paper is devoted to the use of generative adversarial neural networks on the example of solving the problem of image intelligent colorization. Intelligent colorization is the process of changing colors of certain objects and details in an image by using mathematical algorithms. Existing solutions to this problem use standard methods, such as overlaying additional layers on highlighted areas in the image, changing the brightness, saturation, and hue levels. However, all of these methods require active involvement and user skills to locate the objects which need to be repainted, and then also need to adjust other metrics such as shadows, brightness, and exposure on the image to make the converted image look natural. However, machine learning technologies enable automated image processing. The paper presents the software system for intelligent colorization based on the work of two adversarial neural networks: the network-generator and the network-discriminator. The first network is a generator network; it performs input image processing in accordance with the purpose set by the user. The objective of this network is to generate data that is as similar as possible to the original data used in training. The second network is a discriminating network; it evaluates the quality of the result by determining the difference between the sample image that sets the desired processing result and the image that the first network generates. This approach provides high quality training for both networks and, accordingly, high quality of the results. At the stage of data preparation, it is proposed to perform image conversion from RGB color model to YUV or LAB color model that allows to achieve better colorization result. The paper also considers the possibility of moving from image processing to multimodal data processing and proposes to represent input multimodal data about objects on an image in the form of a multimodal multilevel ontological model.