Auditory Augmented Reality (AAR) is an emerging technology that enhances user experiences by overlaying spatially appropriate sounds onto a real environment and gives the impression of virtual sounds coexisting with the physical world. A major challenge in achieving a plausible real-time AAR experience is to seamlessly align virtual soundscapes with real-world environments. To address this, we previously introduced the Augmented reality Room Acoustic Estimator (ARAE). However, the earlier versions of the ARAE encountered difficulties in appropriately replicating the room acoustic behavior due to inadequate rendering of early reflections. To overcome these challenges, we implemented two innovative techniques: mesh segmentation to identify dominant acoustic elements and material estimation through deep learning. Our pilot measurement demonstrated that the proposed method better replicates the sound behavior of the physical environment than the one without material estimation in terms of its frequency characteristics, although some discrepancies were also noted. Future developments will concentrate on customizing the material estimation datasets with the intention of inferring real-world acoustic conditions and optimizing the proposed method through subjective evaluations.
https://www.aes.org/e-lib/browse.cfm?elib=22404
Click to purchase paper as a non-member or login as an AES member. If your company or school subscribes to the E-Library then switch to the institutional version. If you are not an AES member and would like to subscribe to the E-Library then Join the AES!
This paper costs $33 for non-members and is free for AES members and E-Library subscribers.
Learn more about the AES E-Library
Start a discussion about this paper!