Empirical determination of the minimum sufficient training sample size for machine learning models at a given error level

Abstract

This article addresses the problem of empirically determining the minimum sufficient training sample size for machine learning regression models in ore sensor sorting systems. The relevance of this topic stems from the significant costs associated with creating representative datasets in the mining industry. The problem lies in the need to transition from a qualitative analysis of the learning curve to a quantitative assessment of the sufficient sample size for a given error level. The aim of the study is to evaluate a hierarchy of approaches: learning curve – parametric power-law extrapolation – GP-based learning-type curve. Methods: 10-fold GroupKFold cross-validation, Bayesian hyperparameter tuning (Optuna), nonlinear regression, Gaussian processes. Results: R² = 0.93 on test folds; the minimum sufficient sample size for RMSE ≤ 12 was estimated in the range of 559–810 observations. Key conclusion: the proposed method allows for a well-founded determination of the threshold beyond which further expansion of the sample ceases to yield practical benefits.