Hybrid Learning Driven by Dynamic Descriptors for Video Classification of Reflective Surfaces

Visual inspection has recently gained increasing importance in the manufacturing industry and is often addressed by means of learning methodologies applied to data obtained from specific lighting and camera system setups. The industrial scenario becomes particularly challenging when the inspection regards reflective objects, which may affect both the data acquisition and the classification decision process, thus limiting the overall performance. In this context, we observe that the dynamics of the reflected light is the key aspect to characterize these surfaces and needs to be accurately exploited to improve the performances of the learning algorithms. To this aim, we propose a combined model-based and data-driven approach designed to detect defects on the reflective surfaces of industrial products, captured as video sequences under coaxial structured illumination. Specifically, a tunable spatial-temporal descriptor of the evolution of the reflected light (Dynamic Evolution of the Light, DEL) is designed and employed within a Hybrid Learning (HL) framework, where the learning process of a Convolutional Neural Network (CNN) is driven by the model-based descriptor. This approach is also extended by adopting the similar in nature descriptor Dynamic Image. The proposed HL solutions are validated against a whole spectrum of state-of-the-art learning procedures and different descriptors. Experiments run on a dataset coming from an actual industrial scenario confirm the ability of DEL to accurately characterize reflective surfaces and the validity of the HL method, which shows remarkably better performance in fault detection even with respect to modern 3D- CNNs with comparable computational effort.