LARGE LANGUAGE MODELS AS A ROUTING LAYER IN MULTI-CHANNEL MESSENGER SYSTEMS
DOI:
https://doi.org/10.34185/1991-7848.itmm.2026.01.087Keywords:
large language models, intent detection, message routing, tool use, middleware, omnichannel, CRM integration, FastAPIAbstract
Businesses that handle customer communication across multiple messenger channels face a recurring operational problem: getting each incoming message to the right handler without human involvement. Rule-based approaches — keyword filters, button menus, static decision trees — break down quickly when users write freely. This paper describes a middleware architecture where a large language model sits between the incoming message stream and the business logic layer, classifying each message by intent and triggering the appropriate action directly. The model returns structured JSON via a tool use mechanism rather than generating free text, which keeps latency predictable and integration straightforward. The system has been deployed commercially, integrated with platforms including PipeDrive, HubSpot, and Zoho CRM, and is actively used by more than one thousand businesses. Routing errors dropped from 35% to 4% after rollout.
References
Brown T. et al. Language Models are Few-Shot Learners. Advances in NeurIPS. 2020. Vol. 33. P. 1877–1901.
Liu P. et al. Pre-train, Prompt, and Predict. ACM Computing Surveys. 2023. Vol. 55. No. 9. P. 1–35.
Schick T. et al. Toolformer: Language Models Can Teach Themselves to Use Tools. NeurIPS. 2023. Vol. 36.
Wei J. et al. Chain-of-Thought Prompting Elicits Reasoning in LLMs. NeurIPS. 2022. Vol. 35. P. 24824–24837.
