Designing sonic interactions in intelligent reality with egocentric audio technologies

Sonic Interactions in Virtual Environments (SIVE) is an emerging field of study that aims to foster the creation of immersive and interactive sonic experiences in Virtual/Augmented Realities (VR/AR). Sound can convey a wealth of information, such as emotion, meaning and narratives, as well as enhance a multimodal experience. However, designing immersive sonic interactions is a complex task that requires a deep understanding of the embodied, situated and enactive aspects of sound in space. This chapter provides an overview of the field of SIVE and discusses the key concepts and challenges involved in designing meaningful sonic interactions. By using the lens of actor-network theory (ANT) and the new technological paradigm of egocentric audio, this chapter will discuss how sonic interactions can be effectively designed in their early stages to improve the sound-driven design in a multimodal perspective. In addition, the chapter proposes a novel theoretical design tool, called the Entanglement eXperience Map (EXMap), which can be used to define different perspectives and dynamic behaviors of actors within immersive sonic experiences. It considers three main concepts introduced by the egocentric audio technologies and their pivotal participation of an auditory digital twin:Â (i) sense of immersion in the VE, (ii)Â coherence of the experience, and (iii) entanglement between the user (real twin) with a cyber-physical system (digital twin). The following case study in the cultural heritage domain guides the reader:Â a user can have a museum visit experience in immersive audio AR, receiving spatialized information about artwork and artifacts based on personal interests, and emotional and psychological states. Personalized sonic interactions aim to uniquely create a meaningful and memorable experience for the user.