Video Game Music (VGM) is essential to video games. In particular, preparing a suitable VGM is required to raise the quality of the game. However, it is not easy to retrieve a VGM that has both the worldview of the game and the atmosphere of the scene at the same time. Therefore, we propose a system to retrieve VGM considering these conditions. This system receives three inputs. (1) Source VGM MP3: MP3 of VGM to be adopted in the created game, (2) Source VGM Tag: Tag of the scene using the adopted VGM, (3) Target VGM Tag: Tag of another scene to add VGM. First, the variational auto-encoder (VAE) converts Source VGM MP3 into the latent variable z. Second, let Pz be the set of latent variables of the VGM corresponding to Source VGM Tag. Let pc be the center of Pz. Similarly, let Qz be the set of latent variables of the VGM corresponding to Target VGM Tag. Let qc be the center of Qz. Third, let z? be the latent variable of Target VGM MP3. The system assumes that z ? pc = z? ? qc. Therefore, z? is computed by z? = z +qc ? pc. Finally, the system retrieves the VGM with latent variables close to z?. We conducted a preliminary experiment to confirm that VAE can train on this system. As a result of training VAE, the loss function of the training data continued decreasing, but the loss function of the test data stopped decreasing. Therefore, VAE likely caused overtraining. It is necessary to tune the VAE to the task. As a result of the restoration, VAE restored the timbre, not the tempo. It is necessary to use robust audio features for tempo.
https://www.aes.org/e-lib/browse.cfm?elib=22397
Click to purchase paper as a non-member or login as an AES member. If your company or school subscribes to the E-Library then switch to the institutional version. If you are not an AES member and would like to subscribe to the E-Library then Join the AES!
This paper costs $33 for non-members and is free for AES members and E-Library subscribers.
Learn more about the AES E-Library
Start a discussion about this paper!