Multisensory integration in percussion performance

We investigated how auditory and haptic information about the hardness of objects is integrated for the purpose of controlling the velocity with which we strike an object. Our experimental manipulations and data analyses considered a variety of factors that should be integrated in a theory of multisensory perception: expertise of the perceiver; context (unimodal vs. multimodal); between-modalities congruence; between-participants agreement in sensory weighting; performance. On a trial, participants struck a virtual object with a target constant velocity and received feedback on correctness. When the performance criterion was reached, feedback was eliminated, the auditory and/or haptic hardness of the struck object were changed, and the effects on subsequent striking velocity and performance were measured. In unimodal trials only the haptic or auditory display was presented. In multisensory trials, the audio-haptic changes could be congruent (e.g., both increased in hardness) or incongruent. We recruited participants with different levels of expertise with the task: percussionists, nonpercussionist musicians and nonmusicians. For both modalities, striking velocity increased with decreasing hardness, and vice versa. With the vast majority of participants, changes in haptic hardness were perceptually more relevant because they influenced striking velocity to a greater degree than changes in auditory hardness. The perceptual weighting of auditory information was robust to context variations (unimodal vs. multimodal), independent of expertise, uniform across participants and modulated by audio-haptic congruence. The perceptual weighting of haptic information was modulated by context and expertise, more varied across participants and robust to changes in audio-haptic congruence. Performance in tracking velocity was more strongly affected by haptic than auditory information, was not at its best in a multisensory context and was independent of information congruence.