Human motion as a natural control of industrial robots in VR: insights on users’ performance and workload

Recently, the industrial sector has witnessed a massive shift of general interest from the machine to the human, who have become the core of the current industrial evolution. The manifest of the Industry 5.0 is indeed the 'human-centric manufacturing', which places the worker's well-being at the center of the production process. In this framework, a prominent topic is also covered by the increasingly aging workforce, which is bringing particular attention to the senior worker. Indeed, aging brings cognitive and physical decay and is typically related to decreasing flexibility and adaptation to new technologies, which may play a counterposing role to the industrial technological progress. Cognitive research is thus gaining room in all discussions involving workers and machines, and it is supporting the human-centric approach by providing workers' performance and workload assessments. One of the most significant innovations endorsed by Industry 5.0 is leveraging Virtual Reality (VR) for teleoperating or simulating industrial robots. However, VR industrial applications are fairly recent, and workers' psychophysical aspects have been regrettably marginalized until now. Consequently, whether and how VR robot teleoperations are beneficial from human factors' perspective is still under debate, and the role of cognitive science in this respect is now essential. With this Ph.D. thesis, we thus aim to provide a broad overview of performance and workload of users simulating robotic teleoperations in VR. We conducted 5 experimental studies, whose common thread is the industrial robot e-Series UR5e that was here purposely reproduced in VR via Unity. The VIVE Pro Eye VR headset was also deployed in all experiments; it is provided with an integrated eye tracker, which offers an exceptional opportunity for continuous workload monitoring during robotic teleoperation. Furthermore, the last study was conducted at the Berlin Mobile Brain/Body Imaging (MoBI) Laboratory, which provides dedicated tools and approaches for measuring Electroencephalography (EEG) during free motion. The strength of the presented studies thus resides in the combination of multiple metrics for analyzing human behaviors and brain activity during simulated teleoperations, as well as in the assortment of knowledge coming from cognitive science, human factors, human-robot interaction and computer science sectors. Overall, our results significantly contribute to the state of the art on VR-based telerobotics, particularly offering a multimodal and multifaceted overview of human performance and workload when guiding an industrial robotic arm in VR.