Painting recognition with smartphones equipped with inertial measurement unit

Recently, several works have been proposed in the literature to take advantage of the diffusion of smartphones to improve people experience during museum visits. The rationale is that of substituting traditional written/audio guides with interactive electronic guides usable on a mobile phone. Augmented reality systems are usually considered to make the use of such electronic guides more effective for the user. The main goal of such augmented reality system (i.e. providing the user with the information of his/her interest) is usually achieved by properly executing the following three tasks: recognizing the object of interest to the user, retrieving the most relevant information about it, properly presenting the retrieved information. This paper focuses on the first task: we consider the problem of painting recognition by means of measure-ments provided by a smartphone. We assume that the user acquires one image of the painting of interest with the standard camera of the device. This image is compared with a set of reference images of the museum objects in order to recognize the object of interest to the user. Since comparing images taken in different conditions can lead to unsatisfactory recognition results, the acquired image is typically properly transformed in order to improve the results of the recognition system: first, the system estimates the homography between properly matched features in the two images. Then, the user image is transformed accordingly to the estimated homography. Finally, it is compared with the reference one. This work proposes a novel method to exploit inertial measurement unit (IMU) measurements to improve the system performance, in particular in terms of computational load reduction: IMU measurements are exploited to reduce both the computational burden required to estimate the transformation to be applied to the user image, and the number of reference images to be compared with it.