We describe the first system for physically-based wave acoustics including diffraction effects within a holographic experience shared by multiple untethered devices. Our system scales across standalone mobile-class devices, from a HoloLens to a modern smart phone. Audio propagation in real-world scenes exhibits perceptually salient effects that complement visuals. These include diffraction losses from obstruction, re-direction (“portaling”) of sounds around physical doorways and corners, and reverberation in complex geometries with multiple connected spaces. Such effects are necessary in mixed reality to achieve a sense of presence for virtual people and things within the real world, but have so far been computationally infeasible on mobile devices. We propose a novel cloud-enabled system that enables such immersive audio-visual scenarios on untethered mixed reality devices for the first time.

Authors: Chemistruck, Michael; Storck, Kyle; Raghuvanshi, Nikunj
Affiliations: Microsoft Mixed Reality; Microsoft Research(See document for exact affiliation information.)
AES Conference: 2020 AES International Conference on Audio for Virtual and Augmented Reality (August 2020)
Paper Number: 3-5
Publication Date: August 13, 2020 Import into BibTeX
Permalink: https://www.aes.org/e-lib/browse.cfm?elib=20882

Download Now (14.6 MB)

This paper is Open Access which means you can download it for free.

Learn more about the AES E-Library

E-Library Location:   /conf/2020/avar/2020_avar_paper_25.pdf

Start a discussion about this paper!