Method for assessing the condition of an engineering structure element using explanatory artificial intelligence and multimodal data

Abstract

Assessing the technical condition of engineering structural elements is an important task for ensuring operational safety, reliability, and timely maintenance. Conventional inspection approaches are often labor-intensive, time-consuming, and dependent on expert judgment. This study proposes an intelligent approach for structural condition assessment based on machine learning and explainable artificial intelligence (XAI) using multimodal data that combine visual and environmental information. In the developed approach, the condition of a structural surface is evaluated using three input features: the defect area ratio (DefectRatio), humidity, and temperature. Based on these features, the structural state is automatically classified into three condition categories: Normal, Warning, and Critical. To solve the classification task, several machine learning algorithms were investigated, including Random Forest, XGBoost, LightGBM, Support Vector Machine (SVM), and Logistic Regression, with the application of hyperparameter optimization and cross-validation. Experimental results showed that Random Forest and XGBoost achieved the highest classification accuracy on the test dataset (0.90), while Logistic Regression, SVM, and LightGBM demonstrated lower performance, with accuracies of 0.80, 0.70, and 0.60, respectively. The obtained results indicate that the selected feature set is sufficiently informative for distinguishing between different structural condition states, while also revealing differences in the generalization ability of the investigated models. To improve transparency and interpretability, the proposed approach incorporates explainable artificial intelligence techniques, including Permutation Importance, SHAP-based analysis, and rule-based explanations. These methods made it possible to identify the most influential features and explain the reasoning behind model predictions. The analysis confirmed that DefectRatio is the dominant factor in structural condition assessment, whereas humidity and temperature play a supporting contextual role. The proposed approach was implemented in Python using libraries such as NumPy, Pandas, Matplotlib, Scikit-learn, XGBoost, LightGBM, and SHAP. The obtained results demonstrate the feasibility of combining machine learning, multimodal analysis, and XAI for interpretable and automated structural condition monitoring, which can be useful for preventive diagnostics, risk assessment, and decision support in engineering applications..