In virtual reality (VR), 360º video services provided through head mounted displays (HMDs) or smartphones are widely used. Despite the fact that the user’s viewpoint seamlessly changes, sounds through the headphones are fixed even when images change in correspondence with user head motion in many 360º video services. We have been studying acoustic immersion technology that is achieved by, for example, generating binaural sounds corresponding to the user head motion. Basically, our method is composed of angular region-wise source enhancement using array observation signals, multichannel audio encoding based on MPEG-4 Audio Lossless Coding (ALS), and binaural synthesizing of enhanced signals using head related transfer functions (HRTFs). In this paper, we constructed a smartphone-based real-time system for streaming/viewing 360º video including acoustic immersion and evaluated it through subjective tests.
Click to purchase paper as a non-member or login as an AES member. If your company or school subscribes to the E-Library then switch to the institutional version. If you are not an AES member and would like to subscribe to the E-Library then Join the AES!
This paper costs $33 for non-members and is free for AES members and E-Library subscribers.