With the widespread use of smartphones that have multiple sensors and sound processing capabilities, there is a great potential for increased audience participation in music performances. This paper proposes a framework for participatory mobile music based on mapping arbitrary accelerometer gestures to sound synthesizers. The authors describe Handwaving, a system based on neural networks for real-time gesture recognition and sonification on mobile browsers. Based on a multiuser dataset, results show that training with data from multiple users improves classification accuracy, supporting the use of the proposed algorithm for user-independent gesture recognition. This illustrates the relevance of user-independent training for multiuser settings, especially in participatory music. The system is implemented using web standards, which makes it simple and quick to deploy software on audience devices in live performance settings.

Authors: Roma, Gerard; Xambó, Anna; Freeman, Jason
Affiliations: University of Huddersfield, Huddersfield, UK; Queen Mary University of London, London, UK; Georgia Institute of Technology, Atlanta, GA, USA(See document for exact affiliation information.)
JAES Volume 66 Issue 6 pp. 430-438; June 2018
Publication Date: June 18, 2018 Import into BibTeX
Permalink: https://www.aes.org/e-lib/browse.cfm?elib=19582

Click to purchase paper as a non-member or login as an AES member. If your company or school subscribes to the E-Library then switch to the institutional version. If you are not an AES member and would like to subscribe to the E-Library then Join the AES!

This paper costs $33 for non-members and is free for AES members and E-Library subscribers.

Learn more about the AES E-Library

E-Library Location: (CD JAES66)   /jaes66/6/pg430.pdf

DOI: https://doi.org/10.17743/jaes.2018.0026

Start a discussion about this paper!