2-D/3-D fusion-based robust pose normalisation of 3-D livestock from multiple RGB-D cameras

Recently, 3-D scanning data are becoming increasingly important for precision livestock farming. In particular, RGB-D (red, green, blue – depth) data of livestock have come to play a critical role in the field of livestock body measurement. However, the latest livestock pose normalisation methods rely on purely 3-D geometric data and are therefore prone to errors due to noise and missing data. To achieve adequate performance, particularly for different livestock species in practical applications a 2-D/3-D fusion-based robust livestock pose normalisation method is proposed. Firstly, based on advanced 2-D object detection techniques, the proposed approach makes the best use of 2-D information to determine the accurate orientation of livestock in 3-D. Secondly, the 2-D detection results are used to generate frustums in 3-D space to locate livestock targets, which markedly reduces the search space and improves segmentation. Finally, based on a bilateral symmetry-based pose normalisation framework, a more robust pose normalisation algorithm is applied. Compared to existing pose normalization methods that operate in 3-D, extensive experiments with multiple view RGB-D data of livestock show that the proposed method is more robust and practical than existing methods. The proposed algorithm provides pose normalisation in an automatic body measurement system for livestock. This study proposes that the idea that 2-D/3-D fusion-based strategies in 3-D should be explored in more detail, particularly for cases in which the 3-D input captured by consumer designed RGB-D cameras which are often noisy and miss values at certain pixels. All the training databases and codes used in the study can be downloaded freely.