Evaluation of the efficiency of models of deep learning networks increasing the resolution of images

Authors

  • Anatoliy Demchyshyn

DOI:

https://doi.org/10.34185/1562-9945-5-124-2019-15

Keywords:

мережа глибинного навчання, згорткова нейронна мережа, роздільна здатність зображення, гістограма, ентропія, LZ77

Abstract

Since a decade, deep learning networks have shown a significant advantage over other algorithms of computer vision. The paper considers a task of evaluating the efficiency of increasing information content (self-information) of images generated using the two latest models of neural networks: the Residual Dense Network (RDN) for Image Super-Resolution and Enhanced Super-Resolution Generative Adversarial Network (ESRGAN). The increase in information content occurs in a context of creating the image with a fourfold resolution in comparison to the original image. The low resolution test sample is prepared using a decimation algorithm. The algorithm preserves only every n-th pixel of the high resolution original image. The way the low resolution image is prepared simulates the aliasing effect very well. Aliasing effect is a typical image artifact inherent to analog-to-digital conversion.
Usually the quality (distortion) of the image is evaluated basing on values of Peak Signal Noise Ratio and Structure SIMilarity (so called full-reference metrics that perform comparison against etalon image). At the same time the image reconstructed by the deep learning network may subjectively differ from the original sample. The paper uses three no-reference metrics to measure the information content of the image: a histogram displaying the probability density function of pixel shades; entropy value characterizing the degree of diversity of each individual pixel in the sample; and a channel capacity value needed for transmitting the message compressed using LZ77 algorithm. These metrics relate to the so-called "objective".
Acquired results indicate an increase of Shannon entropy when using deep learning networks of the ESRGAN and RDN models from 7.43 to 7.81 and 7.62 respectively. At the same time, it is shown that classical bilinear interpolation of the sample can be successfully applied to increase information content (7.62). It has been demonstrated that LZ77 metric measuring the capacity of a channel for transmitting a message is more adequate for assessing visual quality of the message than Shannon entropy metric.

References

The History of Artificial Intelligence / C. Smith, B. McGuire, T. Huang, G.Yanhg // History of Computing, Univ. of Washington. – 2006. – 27 p.

50 Years of Artificial Intelligence / M. Lungarella, F. Iida, J. Bongard, R.Pfeifer // Essays Dedicated to the 50th Anniversary of Artificial Intelligence. – Springer, 2007. – 399 p.

A Mathematical Theory of Communication / C. Shannon // Reprinted with corrections from The Bell System Technical Journal. – 1948. – Vol. 27. –

P. 379–423, 623–656.

NVIDIA Reinvents Computer Graphics with Turing Architecture [Internet resource] // Gail Laguna. – 2018. Rezhym dostupu do resursu: https://nvidianews.nvidia.com/news/nvidia-reinvents-computer-graphics-with-turing-architecture

Hnatushenko V.V. Rekonstruktsiia rastrovykh zobrazhen na osnovi yikh invariantnoi heometrychnoi modeli / V.V. Hnatushenko, O.V. Reuta // Prykladna heometriia ta inzhenerna hrafika / Pratsi Tavriiskoi derzhavnoi ahrotekhnichnoi akademii. – Melitopol: TDATA, 2004. – Issue. 4, vol. 24. – P.99–102.

Korchynskyi V.M. Analiz metodiv identyfikatsii heometrychnykh form na fotohrammetrychnykh tsyfrovykh zobrazhenniakh dystantsiinoho zonduvannia Zemli / V.M. Korchynskyi, O.V. Spirintseva // Prykladna heometriia ta inzhenerna hrafika / Pratsi Tavriiskoho derzhavnoho ahrotekhnichnoho universytetu. – Melitopol: TDATU, 2014. – Issue. 4, vol.58. – P. 50–57.

Hnatushenko V.V. Otsinka efektyvnosti heometrychnykh i radiometrychnykh peretvoren bahatokanalnykh tsyfrovykh zobrazhen / V.V.Hnatushenko, O.O. Safarov // Prykladna heometriia ta inzhenerna hrafika / Pratsi Tavriiskoho derzhavnoho ahrotekhnichnoho universytetu. – Melitopol: TDATU, 2014. – Issue. 4, vol. 58. – P. 29–33.

Hordiienko O.M. Syntez rastrovykh proektsiinykh zobrazhen na osnovi sumishchennia prostorovykh ta spektralnykh skladovykh / O.M. Hordiienko // Prykladna heometriia ta inzhenerna hrafika / Pratsi Tavriiskoi derzhavnoi ahrotekhnichnoi akademii. – Melitopol: TDATA, 2004. – Issue. 4, vol. 28. – P.109-113.

Tearing apart Googles TPU 3.0 AI coprocessor [Internet resource] // Paul Teich. – 2018. Source: https://www.nextplatform.com/2018/05/10/tearing-apart-googles-tpu-3-0-ai-coprocessor/

Residual Dense Network for Image Super-Resolution [Internet resource] / Yulun Zhang, Yapeng Tian, Yu Kong and others // arXiv. – 2018. – Source: https://arxiv.org/abs/1802.08797.

ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks [Elektronnyi resurs] / Xintao Wang, Ke Yu, Shixiang Wu ta in. // arXiv. – 2018. Source: https://arxiv.org/abs/1809.00219.

Image quality assessment: from error visibility to structural similarity / Zhou Wang, A. Bovik, H. Sheikh, E. Simoncelli // IEEE Transactions on Image Processing. – 2004. – vol. 13, no. 4. – P. 600–612.

Wang Z. Multiscale structural similarity for image quality assessment / Z.Wang, E.P. Simoncelli, A.C. Bovik // Conference Record of the Thirty-Seventh Asilomar Conference on Signals, Systems and Computers. – 2004. – Vol. 2. – P. 1398–1402.

Starovoitov V.V. Indeks SSIM ne iavliaetsia metrikoi i plokho otsenivaet skhodstvo izobrazhenii / V. V. Starovoitov // Sistemnii an. i prikladnaia informatika. – Minsk: BNTU, 2019. – № 2. – P. 12–17.

Avinery R. Universal and Accessible Entropy Estimation Using a Compression Algorithm / R. Avinery, M. Kornreich, R. Beck // Phys. Rev. Lett. 123, 178102, 2019. – 13p.

Published

2019-11-25