This paper introduces a workflow to record paired motion capture and sound of musical performances using an infrared light-based tracking system. This paper discusses example production cases ranging from short loops to full performances, addressing the production challenges and compromises, substantially different from a regular audio recording process. Motion capture is very versatile and can be applied in many disciplines including musical pedagogy, distributed performances, machine learning, and as creative assets. Paired audio-mocap data can be rendered into similar visual content for interactive virtual environments, live performances or fixed media. This extensible workflow can be adapted and extended for different purposes in musical performances to promote further research and creative content development.
https://www.aes.org/e-lib/browse.cfm?elib=20921
Click to purchase paper as a non-member or login as an AES member. If your company or school subscribes to the E-Library then switch to the institutional version. If you are not an AES member and would like to subscribe to the E-Library then Join the AES!
This paper costs $33 for non-members and is free for AES members and E-Library subscribers.
Learn more about the AES E-Library
The Engineering Briefs at this Convention were selected on the basis of a submitted synopsis, ensuring that they are of interest to AES members, and are not overly commercial. These briefs have been reproduced from the authors' advance manuscripts, without editing, corrections, or consideration by the Review Board. The AES takes no responsibility for their contents. Paper copies are not available, but any member can freely access these briefs. Members are encouraged to provide comments that enhance their usefulness.
Start a discussion about this paper!
