Font, Frederic; Stolfi, Ariane; Schroeder, Franziska
Page: 916
Open
Access
The Web MIDI API is intended to connect a browser app with Musical Instrument Digital Interface (MIDI) devices and make them interact. Such an interface deals with exchanging MIDI messages between a browser app and an external MIDI system, either physical or virtual. The standardization by the World Wide Web (W3C) Consortium started about 10 years ago, with a first public draft published on October 2012, and the process is not over yet. Because this technology can pave the way for innovative applications in musical and extra-musical fields, the present paper aims to unveil the main features of the API, remarking its advantages and drawbacks and discussing several applications that could take benefit from its adoption.
Download: PDF (HIGH Res) (1.0MB)
Download: PDF (LOW Res) (297KB)
Open
Access
Nowadays, widely used videoconferencing software has been diffused even further by the social distancing measures adopted during the SARS-CoV-2 pandemic. However, none of the Web-based solutions currently available support high-fidelity stereo audio streaming, which is a fundamental prerequisite for networked music applications. This is mainly because of the fact that the WebRTC RTCPeerConnection standard or Web-based audio streaming do not handle uncompressed audio formats. To overcome that limitation, an implementation of 16-bit pulse code modulation (PCM) stereo audio transmission on top of the WebRTC RTCDataChannel, leveraging Web Audio and AudioWorklets, is discussed. Results obtained with multiple configurations, browsers, and operating systems showthat the proposed approach outperforms theWebRTC RTCPeerConnection standard in terms of audio quality and latency, which in the authors' best case to date has been reduced to only 40 ms between twoMacBooks on a local area network.
Download: PDF (HIGH Res) (3.9MB)
Download: PDF (LOW Res) (845KB)
Many visual programming languages (VPLs), which include Max or PureData, provide a graphic canvas for connecting between functions or data. This canvas, also called a patcher, is basically a graph meant to be interpreted as a dataflow computation by the system. Some VPLs are used for multimedia performance or content generation because the user interface system is generally a significant element of the language. This paper presents a Web-based VPL, JSPatcher, which allows users to build audio graphs using the Web Audio API. Users can use aWeb browser to graphically design and run digital signal processor algorithms using domain-specific languages for audio processing, such as FAUST or Gen, and execute them in a dedicated high-priority thread called AudioWorklet. The application can also be utilized to create interactive programs and shareable artworks online with other JavaScript language built-ins, Web APIs, Web-based audio plugins, or external JavaScript modules.
Download: PDF (HIGH Res) (12.1MB)
Download: PDF (LOW Res) (1.2MB)
Open
Access
Web Audio has a great potential for interactive audio content in which an open standard and easy integration with other web-based tools makes it particularly interesting. From earlier studies, obstacles for students to materialize creative ideas through programming were identified; focus shifted from artistic ambition to solving technical issues. This study builds upon 20 years of experience from teaching sound and music computing and evaluates howWeb Audio contributes to the learning experience. Data was collected from different student projects through analysis of source code, reflective texts, group discussions, and online self-evaluation forms. The result indicates that Web Audio serves well as a learning platform and that an XML abstraction of the API helped the students to stay focused on the artistic output. It is also concluded that an online tool can reduce the time for getting started with Web Audio to less than 1 h. Although many obstacles have been successfully removed, the authors argue that there is still a great potential for new online tools targeting audio application development in which the accessibility and sharing features contribute to an even better learning experience.
Download: PDF (HIGH Res) (4.4MB)
Download: PDF (LOW Res) (364KB)
Open
Access
Advancing knowledge and understanding about performed music is hampered by a lack of annotation data for music expressivity. To enable large-scale collection of annotations and explorations of performed music, the authors have created a workflow that is enabled by CosmoNote, aWeb-based citizen science tool for annotating musical structures created by the performer and experienced by the listener during expressive piano performances. To enable annotation tasks with CosmoNote, annotators can listen to the recorded performances and view synchronized music visualization layers including the audio waveform, recorded notes, extracted audio features such as loudness and tempo, and score features such as harmonic tension. Annotators have the ability to zoom into specific parts of a performance and see visuals and listen to the audio from just that part. The annotation of performed musical structures is done by using boundaries of varying strengths, regions, comments, and note groups. By analyzing the annotations collected with CosmoNote, performance decisions will be able to be modeled and analyzed in order to aid in the understanding of expressive choices in musical performances and discover the vocabulary of performed musical structures.
Download: PDF (HIGH Res) (10.7MB)
Download: PDF (LOW Res) (1.5MB)
This paper presents a non-backend web architecture for generative audio mixing from the Freesound website using a Variational Autoencoder. It is designed to experiment with the nonexisting audios in large audio databases without the need to populate them. It works directly from the browser using JavaScript tools with a serverless approach and relies exclusively on the computational capacity of the client. The platform's Graphical User Interface allows rapid sampling of the autoencoder sound space and is under active development while the logic is finalized. A Variational Autoencoder has been trained to serve as the default model. Users can upload their own to operate independently. The platform aims to provide users with a straightforward and quick-access interface to generative sounds, supporting the audiovisual industry by filling the existing gaps in audio repositories with synthetic media.
Download: PDF (HIGH Res) (2.5MB)
Download: PDF (LOW Res) (440KB)
