TRANSFORMATION OF THE DEBUGGING LEARNING ENVIRONMENT IN THE ERA OF GITHUB COPILOT: HOW TO MAINTAIN LEARNING EFFECTIVENESS?
DOI:
https://doi.org/10.34185/1991-7848.itmm.2026.01.032Keywords:
debugging, information technology, large language models, process management, softwareAbstract
This paper investigates the transformation of software debugging instructional methodologies amidst the active integration of AI assistants. It explores the phenomenon of student «cognitive passivity» and the limitations of current Large Language Models (LLMs) when dealing with complex software systems. A framework for designing «AI-resistant» learning tasks is proposed, leveraging architectural complexity and extensive context volumes to mitigate the potential for automated problem-solving. The study justifies the implementation of a telemetry collection system to monitor student interactions within the Integrated Development Environment (IDE) as a mechanism for verifying academic integrity and task autonomy. Key metrics for distinguishing between human and AI-agent activities are identified, including temporal context-processing parameters and debugging tool usage patterns (breakpoints, stack trace, stepping). Furthermore, the paper describes a prototype toolkit that implements a dual-component assessment model, considering both the correctness of the final code and the methodological integrity of the debugging process. The findings are aimed at developing adaptive educational systems capable of generating personalized recommendations for students.
References
Stasiuk O. L., Khomik O. M., Karpiuk D. R. Analiz pravovykh ryzykiv tsyfrovoho otsiniuvannia v umovakh zmishanoho navchannia. 2025. URL: https://doi.org/10.5281/zenodo.16869553 (date of access: 31.03.2026). [in Ukrainian].
Amoozadeh M. et al. Student-AI Interaction: A Case Study of CS1 students. 2024. URL: https://doi.org/10.48550/arXiv.2407.00305 (date of access: 31.03.2026).
Basha M. et al. CodeWatcher: IDE Telemetry Data Extraction Tool for Understanding Coding Interactions with LLMs. 2025. URL: https://arxiv.org/abs/2510.11536 (date of access: 31.03.2026).
Beleulmi S. Challenges Of Online Assessment During Covid-19 Pandemic: An Experience Of Study Skills Teachers. مجلة آفاق للعلوم. 2022. С. 49. URL: https://doi.org/10.37167/1677-007-002-004 (date of access: 31.03.2026).
Cotroneo D. et al. Human-Written vs. AI-Generated Code: A Large-Scale Study of Defects, Vulnerabilities, and Complexity. 2025. URL: https://doi.org/10.48550/arXiv.2508.21634 (date of access: 31.03.2026).
Denny P. et al. Computing Education in the Era of Generative AI. Communications of the ACM. 2024. URL: https://doi.org/10.1145/3624720 (date of access: 31.03.2026).
Eibl P. et al. Exploring the Challenges and Opportunities of AI-assisted Codebase Generation. 2025. URL: https://arxiv.org/abs/2508.07966 (date of access: 31.03.2026).
Gerlich M. AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking. Societies. 2025. Т. 15, № 1. С. 6. URL:https://doi.org/10.3390/soc15010006
(date of access: 31.03.2026).
Ihantola P. et al. Educational Data Mining and Learning Analytics in Programming. ITICSE '15: Innovation and Technology in Computer Science Education Conference 2015. Vilnius, Lithuania. New York, USA, 2015. URL: https://doi.org/10.1145/2858796.2858798 (date of access: 31.03.2026).
Korpimies K. et al. Unrestricted Use of LLMs in a Software Project Course: Student Perceptions on Learning and Impact on Course Performance. 2024. URL: https://doi.org/10.1145/3699538.3699541 (date of access: 31.03.2026).
Kundu D. et al. Keystroke Dynamics Against Academic Dishonesty in the Age of LLMs. 2024. URL: https://arxiv.org/abs/2406.15335 (date of access: 31.03.2026).
Liang S. et al. The SWE-Bench Illusion: When State-of-the-Art LLMs Remember Instead of Reason. 2025. URL: https://arxiv.org/abs/2506.12286 (date of access: 31.03.2026).
McDanel B., Novak E. Designing LLM-Resistant Programming Assignments: Insights and Strategies for CS Educators. SIGCSE TS 2025. Pittsburgh, USA. New York, USA, 2025. S. 756–762. URL: https://doi.org/10.1145/3641554.3701872 (date of access: 31.03.2026).
Messer M. et al. How Consistent Are Humans When Grading Programming Assignments? 2025. Т. 25, № 4. URL: https://doi.org/10.1145/3759256 (date of access: 31.03.2026).
Pădurean V. et al. BugSpotter: Automated Generation of Code Debugging Exercises. 2024. URL: https://arxiv.org/abs/2411.14303 (date of access: 31.03.2026).
Pitts G. et al. Students’ Reliance on AI in Higher Education: Identifying Contributing Factors. Communications in Computer and Information Science. Cham, 2026. С. 86–97. URL: https://doi.org/10.1007/978-3-032-12773-0_9 (date of access: 31.03.2026).
Shen J. H., Tamkin A. How AI Impacts Skill Formation. 2026. URL: https://arxiv.org/abs/2601.20245 (date of access: 31.03.2026).
Shihab M. et al. The Effects of GitHub Copilot on Computing Students’ Programming Effectiveness, Efficiency, and Processes in Brownfield Coding Tasks. 2025. URL: http://dx.doi.org/10.1145/3702652.3744219 (date of access: 31.03.2026).
Wang Z. et al. How Does Naming Affect Language Models on Code Analysis Tasks? 2024. URL: http://dx.doi.org/10.4236/jsea.2024.1711044 (date of access: 31.03.2026).
Xu Z. et al. CodeVision: Detecting LLM-Generated Code Using 2D Token Probability Maps and Vision Models. 2025. URL: https://arxiv.org/abs/2501.03288 (date of access: 31.03.2026).
Yuan E. et al. Debug-gym: an environment for AI coding tools to learn how to debug code like programmers. 2024. URL: https://www.microsoft.com/en-us/research/blog/debug-gym-an-environment-for-ai-coding-tools-to-learn-how-to-debug-code-like-programmers/ (date of access: 31.03.2026).
