Data smoothing information technology based on criterion of minimum-extent

  • Vovk Serhii

Abstract

For many practical applications, the smoothing problem of data obtained in the presence of noise and anomalous values is relevant. The complexity of solving this problem is due to the fact that the parametric data model is usually unknown, and the presence of anomalous values can cause significant errors. This work is devoted to the development of smoothing data information technology, which is based on the minimum-extent criterion and the smoothness of desired solution. The goal of this paper is to obtain the information technology for smoothing of data distorted by anomalous values and noise.
The data smoothing is the process of data approximation by a smooth function. One of the effective methods for smoothing of data distorted by additive noise is based on the Tikhonov regularization. However, this method is not effective when the data contains the anomal values. To eliminate this drawback, another formulation of the smoothing problem is proposed, which differs by replacing the main quadratic term of the smoothing problem based on Tikhonov regularization by a non-quadratic term formed on the basis of the minimum-extent criterion. A brief form and detailed form of the proposed statement of the smoothing problem are given. It is emphasized that this problem statement leads to the necessity of solving the minimization problem with a non-convex and non-unimodal objective function. Within the proposed information technology framework, the solution of this minimization problem is achieved numerically by the conjugate gradient method, and the data smoothing process is controlled by using tuning parameters, the values of which are set manually or automatically. The proposed information technology has been tested both on data obtained by numerical simulation and on experimental data representing the photoluminescence spectra. The obtained results confirmed the performance of the proposed information technology.

References

Chandola V. Anomaly detection: A survey / V. Chandola, A. Banerjee, V. Kumar // ACM Computing Surveys. – 2009. – V. 41. – N. 3. – P. 15-58.

Lemeshko B.Yu. Rasshirenie oblasti primeneniya kriteriev tipa Grabbsa, ispolzuemyih pri otbrakovke anomalnyih izmereniy / B. Yu. Lemeshko, S. B. Lemeshko // Izmeritelnaya tehnika. – 2005. – N 6. – S.13-19.

Vovk S. M. Kryterii minimumu protiazhnosti / S. M. Vovk // Systemni tekhnolohii. Rehionalnyi mizhvuzivskyi zbirnyk naukovykh prats. Vypusk

(120). – Dnipro, 2019. – C. 19 – 25.

Ermolaev V. A. Metodyi lokalnogo analiza i sglazhivanie vremennyih ryadov i diskretnyih signalov / V. A. Ermolaev, Yu. A. Kropotov. – Matematicheskoe modelirovanie. – 2017. – V. 29. – N. 2. – S. 119–132.

Savitzky A. Smoothing and differentiation of data by simplified least squares procedures / A. Savitzky, M.J. Golay // Analytical Chemistry. – 1964. – N. 36. – P. 1627-1639.

Forsayt Dzh. Mashinnyie metodyi matematicheskih vyichisleniy / Dzh. Forsayt, M. Malkolm, K. Mouler. – M.: Mir, 1980. – 280 s.

Lyubin P.G. Ob odnom metode sglazhivaniya dvumernoy poverhnosti / P. G. Lyubin // Vestnik RUDN. Seriya Matematika. Informatika. Fizika. – 2016. –

N. 2. – S. 37–43.

Vovk S.M. Metod chislennogo differentsirovaniya zashumlennyih dannyih s vyibrosami // RadIoelektronIka, Informatika, upravlInnya. – 2017. – N. 3 –

S. 44-52.

Vovk S. M. General approach to building the methods of filtering based on the minimum duration principle / S.M. Vovk // Radioelectronics and Communications Systems. – 2016. – V. 59. – N. 7. – P. 281-292.

Published
2020-02-24