Opening slide
Sound and touch are innately linked and can, therefore, be explored together in human-computer interactions that involve music and emergent cross-reality (XR) technologies. As the auditory manifestation of sound, vibrations allow us to listen to music not only with our ears but with our whole body too. This simple act of acoustics provides multimodal stimulation to both performers (active) and audiences (passive) alike. At a music performance level, a complex system of auditory—haptic grammar has been established between music, musicians, and audiences that can render subtle differences in the experiences of a musical performance. This presentation explored previous research on the topic of digital musical instruments, audio-tactile feedback, and the relationship between audio-induced vibrations on the body and experiences of music. This serves to build knowledge of the psychophysical systems at work during musical interactions that can be implemented to generate both music and vibration for remote experiences of music performance in virtual reality (VR).
