SPATIO-TEMPORAL FORECASTING OF DIGITAL IMAGES WITH IRREGULAR TIME INTERVALS: A REVIEW OF DEEP NEURAL NETWORK APPROACHES
DOI:
https://doi.org/10.34185/1562-9945-5-166-2026-20Keywords:
digital image processing, continuous-time forecasting, spatio-temporal dynamics, irregular sampling, neural networks, deep learning, Generative Adversarial Networks, Diffusion ModelsAbstract
This study evaluates four deep learning architectures – a Time-Aware ConvLSTM, Continuous Temporal Transformer, Spatio-Temporal GAN, and Continuous Diffusion Model – for forecasting digital image sequences at irregular time intervals. Tested on Moving MNIST and atmospheric climate records using a stochastic continuous-time masking protocol, the results demonstrate that the ConvLSTM and Transformer severely blur predictions of rigid-body trajectories. In contrast, the GAN achieves robust structural fidelity across both datasets. The CDM excels in modeling rigid-body trajectories but struggles significantly with smooth atmospheric gradients. A Transformer-GAN architecture is proposed to provide high-fidelity spatio-temporal forecasting of digital images with irregular time intervals.
References
1. Shi, X., Chen, Z., Wang, H., Yeung, D. Y., Wong, W. K., & Woo, W. C. (2015). Convo-lutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting. Advances in Neural Information Processing Systems, 28.
2. Neil, D., Pfeiffer, M., & Liu, S. C. (2016). Phased LSTM: Accelerating Recurrent Net-work Training for Long or Event-based Sequences. Advances in Neural Information Processing Systems, 29.
3. Cui, Z., Henrickson, K., Ke, R., & Wang, Y. (2019). Traffic Graph Convolutional Long Short-Term Memory Neural Network for Urban Traffic Flow Prediction. Transportation Research Part C: Emerging Technologies, 105, 361-374.
4. Vaswani, A., Shazeer, N., Parmar, N.,Uszkoreit, J., Jones, L.,Gomez, A.N.,& Polo-sukhin,I.(2017).Attention is all you need. Advances in neural information processing systems, 30.
5. Fonseca, A. H. D. O., Zappala, E., Caro, J. O., & Van Dijk, D. (2023). Continuous spatio-temporal transformers. arXiv preprint arXiv:2301.13338.
6. Rußwurm, M., & Körner, M. (2020). Self-Attention for Raw Optical Satellite Time Series Classification. ISPRS Journal of Photogrammetry and Remote Sensing, 169, 421-435.
7. Tipirneni, S., & Reddy, C. K. (2022). Self-Supervised Transformer for Sparse and Irregu-larly Sampled Multivariate Clinical Time Series. ACM Transactions on Knowledge Discovery from Data, 16(6), 1-17.
8. Rubanova, Y., Chen, R. T. Q., Bettencourt, J., & Duvenaud, D. (2018). Neural Ordinary Differential Equations. Advances in Neural Information Processing Systems, 31.
9. De Brouwer, E., Simm, J., Arany, A., & Moreau, Y. (2019). GRU-ODE-Bayes: Continu-ous Modeling of Sporadically-Observed Time Series. Advances in Neural Information Processing Systems, 32.
10. Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2020). Generative Adversarial Networks. Communications of the ACM, 63(11), 139–144. https://doi.org/https://doi.org/10.1145/3422622
11. Xie, Y., Franz, E., Chu, M., & Thuerey, N. (2018). tempoGAN:A Temporally Coherent, Volumetric GAN for Super-resolution Fluid Flow. ACM Transactions on Graphics, 37(4), 1 15.
12. Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., & Poole, B. (2020). Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456.
13. Price, I., Sanchez-Gonzalez, A., Alet, F., et al. (2023). GenCast: Diffusion-based Ensem-ble Forecasting for Medium-Range Weather. arXiv preprint arXiv:2312.15796.
14. Srivastava, N., Mansimov, E., & Salakhutdinov, R. (2015). Unsupervised learning of video representations using LSTMs. Proceedings of the 32nd International Conference on Machine Learning (ICML). http://www.cs.toronto.edu/~nitish/unsupervised_video/
15. Kalnay, E., Kanamitsu, M., Kistler, R., Collins, W., Deaven, D., Gandin, L., & Joseph, D. (2018). The NCEP/NCAR 40-year reanalysis project. Renewable energy, 1, 146-194.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 System technologies
This work is licensed under a Creative Commons Attribution 4.0 International License.
