AES E-Library

G. Burloiu, V. Mihai, and S. Damian, "Layered Motion and Gesture Sonification in an Interactive Installation," J. Audio Eng. Soc., vol. 66, no. 10, pp. 770-778, (2018 October.). doi: https://doi.org/10.17743/jaes.2018.0047 G. Burloiu, V. Mihai, and S. Damian, "Layered Motion and Gesture Sonification in an Interactive Installation," J. Audio Eng. Soc., vol. 66 Issue 10 pp. 770-778, (2018 October.). doi: https://doi.org/10.17743/jaes.2018.0047
Abstract: SoundThimble is an interactive sound installation based on the relationship between human motion and virtual objects in 3D space. A Vicon infrared motion-capture system and custom software are used to track, interpret, and sonify the movement and gestures of a performer relative to a virtual object. The authors explore the resulting possibilities for layered sonification dynamics and extended perception and expression in internal tests as well as in a public demo. Experimental evaluation reveals an average object search time of around 60 s, as well as thresholding ranges for effective gesture spotting. The underlying software platform is open source and portable to similar hardware systems, leaving room for extension and variation. This paper presents the pilot application of the proposed framework. Audience members entering the tracking area shift among the roles of game player, sonic performer, and composer/arranger, according to an iterative interaction schema. The central vehicle in all three layers is the “sound-thimble” itself, a virtual object with particular spatial, sonic, and interaction attributes.

@article{burloiu2018layered,
author={burloiu, grigore and mihai, valentin and damian, stefan},
journal={journal of the audio engineering society},
title={layered motion and gesture sonification in an interactive installation},
year={2018},
volume={66},
number={10},
pages={770-778},
doi={https://doi.org/10.17743/jaes.2018.0047},
month={october},} @article{burloiu2018layered,
author={burloiu, grigore and mihai, valentin and damian, stefan},
journal={journal of the audio engineering society},
title={layered motion and gesture sonification in an interactive installation},
year={2018},
volume={66},
number={10},
pages={770-778},
doi={https://doi.org/10.17743/jaes.2018.0047},
month={october},
abstract={soundthimble is an interactive sound installation based on the relationship between human motion and virtual objects in 3d space. a vicon infrared motion-capture system and custom software are used to track, interpret, and sonify the movement and gestures of a performer relative to a virtual object. the authors explore the resulting possibilities for layered sonification dynamics and extended perception and expression in internal tests as well as in a public demo. experimental evaluation reveals an average object search time of around 60 s, as well as thresholding ranges for effective gesture spotting. the underlying software platform is open source and portable to similar hardware systems, leaving room for extension and variation. this paper presents the pilot application of the proposed framework. audience members entering the tracking area shift among the roles of game player, sonic performer, and composer/arranger, according to an iterative interaction schema. the central vehicle in all three layers is the “sound-thimble” itself, a virtual object with particular spatial, sonic, and interaction attributes.},}

TY - paper
TI - Layered Motion and Gesture Sonification in an Interactive Installation
SP - 770 EP - 778
AU - Burloiu, Grigore
AU - Mihai, Valentin
AU - Damian, Stefan
PY - 2018
JO - Journal of the Audio Engineering Society
IS - 10
VO - 66
VL - 66
Y1 - October 2018 TY - paper
TI - Layered Motion and Gesture Sonification in an Interactive Installation
SP - 770 EP - 778
AU - Burloiu, Grigore
AU - Mihai, Valentin
AU - Damian, Stefan
PY - 2018
JO - Journal of the Audio Engineering Society
IS - 10
VO - 66
VL - 66
Y1 - October 2018
AB - SoundThimble is an interactive sound installation based on the relationship between human motion and virtual objects in 3D space. A Vicon infrared motion-capture system and custom software are used to track, interpret, and sonify the movement and gestures of a performer relative to a virtual object. The authors explore the resulting possibilities for layered sonification dynamics and extended perception and expression in internal tests as well as in a public demo. Experimental evaluation reveals an average object search time of around 60 s, as well as thresholding ranges for effective gesture spotting. The underlying software platform is open source and portable to similar hardware systems, leaving room for extension and variation. This paper presents the pilot application of the proposed framework. Audience members entering the tracking area shift among the roles of game player, sonic performer, and composer/arranger, according to an iterative interaction schema. The central vehicle in all three layers is the “sound-thimble” itself, a virtual object with particular spatial, sonic, and interaction attributes.