Issues of improving the efficiency of embedded control systemswith a computer vision module

Abstract

The development of autonomous robotics, UAVs, and Industry 4.0 requires expanding the sensory capabilities of cyber-physical systems through the implementation of computer vision (CV) modules on edge devices [1, 2]. The transfer of CV algorithms is implemented via Edge Computing and TinyML paradigms, utilizing spatio-temporal optimization techniques such as quantization (transition from FP32 to INT8/INT4) and synaptic pruning [3, 4].

Global studies indicate that any compression requires a trade-off between accuracy and latency, and also makes models less robust to external disturbances [5]. Domestic scientists have significant achievements in the fields of machine learning, monitoring, and image compression; however, these issues are primarily considered in the context of offline analysis or data transmission [6, 7].

In the scientific community, there is an inconsistency of approaches: Data Science specialists evaluate models in isolation using static metrics and ignore the impact of compressed network noise on the control system [8]. Meanwhile, automatic control specialists perceive the video sensor as a classic link with deterministic delay, which may not correspond to the stochastic nature of inference [9].

Finding the optimal trade-off between the computational efficiency of the neural network (inference latency) and the performance quality of the automatic control system itself to achieve maximum reliability and efficiency of autonomous devices.

Integrating modern computer vision algorithms into resource-limited edge devices leads to a fundamental contradiction.

In classical automatic control theory, a video sensor acts as a dynamic link of pure delay, which creates a phase lag and linearly reduces the system's phase stability margin [10]. Exceeding the delay limit inevitably leads to a loss of controllability of the object. To avoid this, developers use aggressive compression to reduce inference time, which restores the phase margin but distorts error statistics [11]. Specifically, bit-depth reduction generates quantization noise (high-frequency jitter), which acts extremely destructively on the differential component of the regulator and causes chaotic oscillations of the actuators [12].

Since classic optimization metrics are not suitable for such cases, the authors propose a comprehensive multi-level methodology (Co-design) that allows integrating compression parameters into the dynamic’s equations. It encompasses six sequential tasks: creating a digital twin using the MuJoCo physics engine in Unity or NVIDIA Isaac Sim environments [13, 14]; synthesis of ideal control to obtain reference data; Vision-in-the-loop simulation with an uncompressed network; optimization and emulation in QEMU/Renode for TinyML testing [15]; hardware implementation of Hardware-in-the-loop on Nvidia Jetson/Raspberry Pi/STM32 controllers; and a final comparative analysis of the impact of compression on overshoot and stability margins.

The conducted analysis suggests that increasing the efficiency of embedded ACS with computer vision is a complex interdisciplinary task. Optimizing neural networks solely by machine learning metrics, without considering the physical object's dynamics, is not effective and can destabilize the control loop.

The proposed approach, which combines Data Science and control theory methods (from simulation to HIL testing), creates a solid foundation for researching the relationship between hardware compression parameters and control quality indicators. In the future, its implementation will help develop more accurate mathematical descriptions of the control loop, considering variable delay and quantization noise, as well as formulate clear engineering recommendations for developers of autonomous systems.