Review of natural language processing methods: evolution, architectures, and application domains

Abstract

The global Natural Language Processing (NLP) market demonstrates rapid, almost exponential growth, reflecting immense commercial interest and the digital economy's reliance on AI algorithms. However, the rapid scaling of neural network parameters—from millions to hundreds of billions—has introduced new computational and ecological challenges. Furthermore, models inevitably inherit biases present in training data, raising critical ethical and safety concerns. There is an urgent need for novel, highly efficient architectures and transparent optimization methods to overcome these limitations.

The primary goal of this research is to conduct a comprehensive analytical review of NLP methods, trace their architectural evolution from traditional statistical machine learning models to modern Large Language Models (LLMs) and Large Reasoning Models (LRMs), and systematize their practical application domains at the current stage of technological development.

The evolution of NLP is conceptually divided into three fundamental paradigms: rule-based symbolic systems, statistical methods (e.g., TF-IDF, SVM, n-grams), and deep learning. A tectonic shift occurred with the introduction of the Transformer architecture, which replaced recurrent blocks with self-attention mechanisms, enabling parallel processing. To address the quadratic computational complexity of processing long contexts, the study analyzes non-traditional optimization approaches, primarily Neuroevolution and Neural Architecture Search (NAS), which automatically construct optimal transformer blocks. The paper also systematizes the practical application of NLP in healthcare, software engineering, and business analytics, demonstrating varying degrees of accuracy and highlighting the necessity of Retrieval-Augmented Generation (RAG) technologies.

NLP has become the most dynamic segment of artificial intelligence. A fundamental shift from statistical imitation to algorithmic reasoning is observed, evidenced by LRMs like DeepSeek-R1, which are capable of self-verifying logic and mitigating factual hallucinations. As models approach the limits of silicon architecture, the future lies in combining neural networks with evolutionary algorithms. Future research must focus on overcoming ethical challenges, ensuring digital equality for low-resource languages, and advancing neurosymbolic AI and quantum computing adaptations.