AES New York 2017
Game Audio & VR Track Event Details
Wednesday, October 18, 9:30 am — 10:30 am (Rm 1E06 - PMC Room)
Spatial Audio: SA19 - Sound Design in 3D for Dance Music
Presenter:Lasse Nipkow, Silent Work LLC - Zurich, Switzerland
Producers of Dance Music pursue the target to put
nightclub audiences into trance.
3D audio includes a high
potential to increase sound impressions significantly and engage the emotions, as is the case in film sound. 3D audio allows to create spectacular spatiality and sound sources around the listener. There are however some challenges: Clubbers on the dance floor are not oriented in the same way as listener of classical music in the concert hall because they are dancing. PA systems of music clubs are nearly exclusively designed for mono and stereo reproduction. Therefore, the impact by changing to a 3D audio system is significant.
During the presentation, the possibilities of musical design in consideration of psychoacoustic phenomena will be described and demonstrated with example recordings and videos.
Wednesday, October 18, 10:15 am — 11:15 am (Rm 1E13)
Game Audio & VR: GA01 - Will VR Be a Game Changer? Challenges & Opportunities in 6DoF Audio
Presenter:Brooklyn Earick, G'Audio Lab - Los Angeles, CA, USA
Panelists:
Andrew Grathwohl, Director of Media Technology, Littlstar
Jean-Marc Jot, Magic Leap - Sunnyvale, CA, USA
Adam Levenson, VP Business Development, Krotos
6DoF(degrees of freedom) type of VR content allows users the ultimate freedom to explore the virtual world, but it is a total pain in the neck for content creators including audio engineers. Compared to 360 videos, 6DoF experiences just require so much more to consider — room simulation, all possible interactive scenarios, and even spatialization that is specially crafted for full VR. This session will explore the major challenges audio engineers are facing in 6DoF content creation and how they are being addressed. Panels will share technologies they are embracing and where these applications are heading. The session intends to provide insights for spatial audio design in both gaming VR and non-gaming VR.
Wednesday, October 18, 3:45 pm — 4:45 pm (Rm 1E13)
Game Audio & VR: GA02 - VR Audio: Workflows and Background for a New Interactive Audio Paradigm
Presenter:Sally-Anne Kellaway, OSSIC - San Diego, CA, USA
Audio for Virtual Reality (VR) presents a significant array of challenges and augmentations to the traditional requirements of sound designers employed within the video games industry. The change in perspective and embodiment of the player, as well as the new technologies involved requires the employment of additional tools and consideration of object size, spacing, and spatial design as a more significant part of the sound design process. We will unpack some of the new technologies and potential approaches to these tasks from the perspective of developing audio for all scales of interactive VR applications. Focusing on the design considerations and processes required in this unique medium, the content of this presentation is designed to give insight in to the complexity of designing for this new medium, and potential pipeline and work flow approaches.
AES Members can watch a video of this session for free.
Wednesday, October 18, 5:00 pm — 6:00 pm (Rm 1E13)
Game Audio & VR: GA03 - Immersive Audio & VR/AR: A Traveler’s Guide to the Growing Landscape of Experiential Audio
Presenters:Josh Antonuccio, Ohio University - Athens, OH, USA
Richard Warp, Intonic - Emeryville, CA, USA
Audio is going to be essential for creating immersive experiences, but what should the aspiring producer or engineer learn in order to excel, and how is higher education responding to the skills gap in this new field? This session is led by Josh Antonuccio, lecturer at the School of Media Arts and Studies at Ohio University and co-creator of the Immersive Media Initiative, and Richard Warp, founder of Intonic and VR audio professional. Topics include workflows and technology for immersive platforms, considerations for working with spatial audio, and the growing need for audio development expertise within augmented and virtual reality. Enjoy a fascinating “fireside” discussion covering educational, technical, and professional aspects of producing audio in the domain of “the 3 Rs”–VR, AR, and MR.
AES Members can watch a video of this session for free.
Thursday, October 19, 9:00 am — 10:00 am (Rm 1E13)
Game Audio & VR: GA04 - Gateways to the East — Music and Recording Techniques for Chinese Video Games
Presenter:Fei Yu, Dream Studio - Los Angeles/ Beijing
As the Music Supervisor for NetEase online game Revelation, Fei Yu would love to discuss the challenge and triumphs of communicating and collaborating from across the world, the technology of recording for Chinese- Western style music, and the creative process, strategies for working with ethnic soloists to create authentic world sound and recording strategies for combining orchestra with soloists. The soundtrack became one of the first game scores to be released by the prestigious film music label Varese Sarabande and was Winner of the BSOSpirit Jerry Goldsmith Award and the Tracksounds Genius Choice Vote for Best Score: Video Game. Nominated for an International Film Music Critics Association Award, A Tracksounds Cue Award, A Reel Music Award, A Game Audio Network Guild Award and named one of the top ten scores of 2015 (game and film) by Cinematic Sound Radio.
AES Members can watch a video of this session for free.
Thursday, October 19, 9:30 am — 10:30 am (Rm 1E10)
Game Audio & VR: GA11 - Creating Interactive, Reactive, and Generative Music with Pure Data
Presenter:Yuli Levtov, Reactify - London, UK
Since the advent of libpd, Pure Data, Max’s open-source and slightly more "bare-bones" cousin, has found its way into many embedded production applications, from mobile apps to Web and VR experiences. Furthermore, bringing procedural and interactive music to platforms such as Unity and the Web has recently been made even more practical by a new and powerful tool, Heavy, which converts Pure Data patches into highly-performant code that can run on many different platforms. Reactify is a London-based creative production company focusing on the possibilities of interactive, reactive, and generative music, with Pure Data sitting at the core of almost all of its projects. In this presentation, co-founder Yuli Levtov will give an overview of how Pure Data can be used to interface with physical or virtual reality in creative ways, and provide a platform for collaboration between developers and artists, and how Heavy can close the gap between a rapid-prototyping tool and efficient production environment.
AES Members can watch a video of this session for free.
Thursday, October 19, 10:45 am — 12:15 pm (Rm 1E13)
Game Audio & VR: GA05 - Star Trek: Bridge Crew VR - Audio Post Mortem
Presenters:Justin Drust, Red Storm Entertainment - Cary, NC, USA
Matthew McCallus, Expert Sound Programmer, Red Storm Entertainment
Look, listen, and learn from the Red Storm Audio Team behind Star Trek: Bridge Crew as they show us new frontiers in VR game audio development from multiple perspectives—the Audio Director, Audio Programmer and others. Discover the tips, tricks, and techniques of the audio design process from conception to completion.
Thursday, October 19, 1:30 pm — 3:00 pm (Rm 1E13)
Game Audio & VR: GA06 - AR/VR: New York Initiatives and Directions
Chair:Agnieszka Roginska, New York University - New York, NY, USA
Panelists:
Kai Falkenberg, First Deputy Commissioner of the NYC Mayor's Office of Media and Entertainment
Justin Hendrix, NYC Media Lab
Erik Hockman, MSG Ventures
Jeremy Rowe, New York University - New York, NY, USA
The session focuses on the latest initiatives and developments of Augmented and Virtual Reality research and development centers in NYC. Panelists will discuss, share, and give specific examples of some of the projects and directions currently explored in AR & VR, including initiatives from NYC’s Mayor’s office, the NYC Media Lab, the NYU Holodeck project, and Madison Square Garden Ventures.
Thursday, October 19, 1:30 pm — 3:00 pm (Rm 1E06 - PMC Room)
Game Audio & VR: GA07 - Spatial Sound for Console and PC Games
Presenters:Robert Ridihalgh, Microsoft - Redmond, Washington, USA
Scott Selfon, Microsoft - Redmond, WA, USA
Mark Yeend, Creative Director, ATG at Xbox at Microsoft
In Spring 2017, Xbox One and Windows added native platform-level support for spatial sound: games and other applications could now implement audio above, below, and around the listener once, and transparently render to multiple endpoints (speakers, headphones) and formats (Dolby Atmos, Windows Sonic for Headphones), while continuing to support all playback devices and mixing of “legacy” non-spatial content. This talk covers some of the early learning that has emerged from these efforts in terms of: spatial sound design and music best practices for both mixed reality and screen-based scenarios; audio creation and mixing pipelines for games; how games are combining channel-based and dynamic sound mixes; performing listening quality assurance with an often dizzying array of output and downmixing options; and more. Throughout we’ll highlight some impactful and common use cases as exercised by early titles, from both technical and aesthetic perspectives.
Thursday, October 19, 3:00 pm — 4:00 pm (Rm 1E09)
Spatial Audio: SA06 - Perceptual Thresholds of Spatial Audio Latency for Dynamic Virtual Auditory Environments
Presenter:Ravish Mehra, Oculus Research - Redmond, WA, USA
Generating the acoustic signals that reproduce the properties of natural environments through headphones remains a significant technical challenge. One hurdle is related to the time it takes to update the signal each time the observer moves. The end-to-end spatial audio latency (SAL) is the time elapsed between the listener assuming a new position and the updated sound being delivered to their ears. It is comprised of latencies in head-tracking, HRTF interpolation and filtering, operating system callback, audio driver and hardware (D/A conversion) buffering, and other parts of the signal processing chain. Because SAL is currently inevitable, it is important to know what SAL is detectable to set minimum thresholds for SAL in virtual auditory environments.
We used a 2-interval-forced-choice paradigm to measure SAL detectability at (10 and 60 degree) azimuths, both with and without the presence of co-located visual stimuli. Overall, mean SAL thresholds were between 128ms and 158ms. Consistent with results from minimum audible motion angle data, thresholds were greater at larger azimuthal positions. A retrospective analysis revealed that listeners who strategically varied the velocity, acceleration and rate of their head rotation were better able to perform the task. This suggests that thresholds for SAL will be lower for applications where users are expected to move their heads more rapidly and abruptly. Results are discussed in the context of prior research and the potential implications for rendering Virtual Reality audio.
Thursday, October 19, 3:15 pm — 4:15 pm (Rm 1E13)
Game Audio & VR: GA08 - Immersive Audio for VR Workflow
Moderator:Michael Kelly, DTS, Inc. - London, UK
Panelists:
Joel Douek, Ecco VR - Los Angeles, CA, USA
Bonnie Bogovich, BlackCatBonifide - Austin, TX, USA
Ken Usami, Capcom - Osaka, Japan
Lidwine Ho, France télévisions - Paris, France
In the emerging field of AR/VR both the tools and techniques for creating content are still in a nascent state. Ways of thinking through the creative process and applications workflows are surfacing and evolving in real-time with new product development, making this a very exciting time to be a content creator (and a content consumer) but also includes potential gaps and problems as standard best practices are only developing.
AES Members can watch a video of this session for free.
Thursday, October 19, 4:30 pm — 6:00 pm (Rm 1E13)
Game Audio & VR: GA09 - Music Recording for Virtual Reality
Chair:Gavin Kearney, University of York - York, UK
Presenters:
Hyunkook Lee, University of Huddersfield - Huddersfield, UK
Hashim Riaz, University of York - York, Yorkshire, UK
Mirek Stiles, Abbey Road Studios - London, UK
There is a growing market for innovative ways to appreciate and listen to music through new Virtual Reality (VR) experiences made accessible through smartphones and VR headsets. However, production workflows for creating immersive musical experiences over VR are still in their infancy. This workshop explores the effectiveness of different microphone configurations and recording techniques in a higher-order Ambisonic processing framework to deliver an engaging and hyper-real interactive VR music experience, with and without immersive 360 visuals. In particular, it will look at how traditional recording workflows can be adapted to immersive 360. Test material will be demonstrated of a live popular music recordings undertaken at Abbey Road with traditional music recording techniques such as spot and stereo microphone setups and advanced techniques using dedicated VR multichannel microphone arrays.
AES Members can watch a video of this session for free.
Friday, October 20, 9:00 am — 11:00 am (Rm 1E13)
Game Audio & VR: GA10 - State of the Art: Kickstart Your Career in Audio for Games, VR, and Beyond
Chair:Scott Looney, Academy of Art University - San Francisco, CA, USA
Panelists:
Brennan Anderson, Senior Audio Producer, Pyramind Studios - San Francisco, CA, USA
Bonnie Bogovich, BlackCatBonifide - Austin, TX, USA
Damian Kastbauer
Larry the O, Toys in the Attic - Vallejo, CA, USA
Make no mistake, game audio is a highly technical business. Creating compelling sound for games takes an incredible combination of skills that goes far beyond writing great music and designing awesome sounds. The best game audio designers possess deep expertise that highlights their dedication to the overall experience, and as the industry matures and moves into more immersive fields like VR and XR, an all-encompassing skillset like this will only become more sought after.
This panel will discuss how to succeed in this field by learning from industry professionals who have spent decades moving the needle in interactive audio and who have built their careers on a philosophy of lifelong learning and intellectual curiosity while breaking new ground at the bleeding edge of their art.
AES Members can watch a video of this session for free.
Friday, October 20, 1:30 pm — 3:00 pm (Rm 1E06 - PMC Room)
Game Audio & VR: GA12 - The “Horrific” Sound of Resident Evil 7: Biohazard
Presenters:Akiyuki Morimoto, Capcom - Osaka, Japan
Ken Usami, Capcom - Osaka, Japan
The audio team behind Capcom’s Resident Evil 7: Biohazard will discuss their methods to create realistic high-quality sound and the systems that allowed more flexibility and efficiency in sound implementation by introducing various automated tools. The team will also discuss about how the music design achieved the horror aspect of sound design and the game. The team chose to use a technique called "music-concrete" to cover a wide variety of sound texture, tones, and noises to deliver an uncomfortable and disturbing atmosphere in the game. They will also share their workflows of crafting the horror scene with sound by collaborating with level designers.
Friday, October 20, 3:15 pm — 4:15 pm (Rm 1E13)
Game Audio & VR: GA13 - VR Analytics Developed from Perceptual Audio Research to Improve Tools and Processes for Developers
Chair:Patrick Flanagan, THX Ltd. - San Francisco, CA, USA
Panelists:
Juan Simon Calle, New York University - New York, NY, USA; THX
Sean Olive, Harman International - Northridge, CA, USA
Nils Peters, Qualcomm, Advanced Tech R&D - San Diego, CA, USA
Nick Zacharov, DELTA SenseLab - Hørsholm, Denmark
Leading experts in perceptual audio testing will discuss perceptual methods used to analyze user behavior in VR experiences. The panel will also discuss ongoing research at NYU that aims to quantize localization behavior that is useful feedback to Developers. They have created a Citizen Science game in a VR environment that can compare HRTF filtersets and spatial audio renderers in order to provide analytics on source accuracy and acquisition speed.
AES Members can watch a video of this session for free.
Friday, October 20, 4:30 pm — 6:00 pm (Rm 1E13)
Game Audio & VR: GA14 - Immersive Headphone Audio Reproduction in VR/AR
Chair:Juergen Peissig, Sennheiser Electronics - Wedemark, Germany
Panelists:
Linda A. Gedemer, Source Sound VR - Woodland Hills, CA USA; University of Salford - Salford, UK
Jean-Marc Jot, Magic Leap - Sunnyvale, CA, USA
Veronique Larcher, Director AMBEO Immersive Audio, Sennheiser - Switzerland
Qing Zhang, HUAWEI European Research Center - Munich, Germany
Major applications of virtual and augmented reality today are presenting with head mounted displays. This allows for mobility and unconstrained movements and is important especially for augmented reproduction to immersively embed the virtual into the real environment. Here headphone audio reproduction is mandatory to reproduce location dependent personalized 3D sound.
Audio rendering has to be of high perceptual quality to yield a high degree of spatial resolution, externalization and congruence of the virtual with the real acoustical environment. Fitting the augmented audio content to the real acoustical environment and the headphone being transparent for the real acoustical environment are important features.
The panelists will discuss various aspects of these challenges for signal processing and headphone acoustics under different AR & VR application requirements.
Saturday, October 21, 1:30 pm — 2:30 pm (Rm 1E13)
Game Audio & VR: GA15 - Create Sound and Music for VR and 360 Using Common DAW Workflows.
Presenter:Tom Ammermann, New Audio Technology GmbH - Hamburg, Germany
This session focuses on the latest initiatives and developments of Augmented and Virtual Reality research and development centers, including the first publicly funded VR/AR Lab in the country – in New York City. Panelists will discuss, share, and give specific examples of some of the projects and directions currently explored in AR/VR.