In virtual reality (VR), 360º video services provided through head mounted displays (HMDs) or smartphones are widely used. Despite the fact that the user’s viewpoint seamlessly changes, sounds through the headphones are fixed even when images change in correspondence with user head motion in many 360º video services. We have been studying acoustic immersion technology that is achieved by, for example, generating binaural sounds corresponding to the user head motion. Basically, our method is composed of angular region-wise source enhancement using array observation signals, multichannel audio encoding based on MPEG-4 Audio Lossless Coding (ALS), and binaural synthesizing of enhanced signals using head related transfer functions (HRTFs). In this paper, we constructed a smartphone-based real-time system for streaming/viewing 360º video including acoustic immersion and evaluated it through subjective tests.

Authors: Niwa, Kenta; Ochi, Daisuke; Kameda, Akio; Kamamoto, Yutaka; Moriya, Takehiro
Affiliations: NTT Media Inelligence Laboratories, Tokyo, Japan; NTT Communication Science Labs, Atsugi-shi, Kanagawa-ken, Japan(See document for exact affiliation information.)
AES Convention: 141 (September 2016) Paper Number: 9676
Publication Date: September 20, 2016 Import into BibTeX
Subject: Spatial Audio and Recording & Production
Permalink: https://www.aes.org/e-lib/browse.cfm?elib=18480

Click to purchase paper as a non-member or login as an AES member. If your company or school subscribes to the E-Library then switch to the institutional version. If you are not an AES member and would like to subscribe to the E-Library then Join the AES!

This paper costs $33 for non-members and is free for AES members and E-Library subscribers.

Learn more about the AES E-Library

E-Library Location:   /conv/141/9676.pdf

Start a discussion about this paper!