This paper presents an Internet of Musical Things ecosystem involving musicians and audiences interacting with a smart mandolin, smartphones, and the Audio Commons online repository Freesound. The ecosystem has been devised to support performer-instrument and performer-audience interactions through the generation of musical accompaniments exploiting crowd-sourced sounds. We present two use cases investigating how audio content retrieved from Freesound can be leveraged by performers or audiences to produce accompanying soundtracks for music performance with a smart mandolin. In the performer-instrument interaction use case, the performer can select content to be retrieved prior to performing through a set of keywords and structure it in order to create the desired accompaniment. In the performer-audience interaction use case, a group of audience members participates in the music creation by selecting and arranging Freesound audio content to create an accompaniment collaboratively. We discuss the advantages and limitations of the system with regard to music making and audience participation, along with its implications and challenges.

Jamming with a Smart Mandolin and Freesound-based Accompaniment

Playsound.space is a web-based tool to search for and play Creative Commons licensed-sounds which can be applied to free improvisation, experimental music production and soundscape composition. It provides a fast access to about 400k non-musical and musical sounds provided by Freesound, and allows users to play/loop single or multiple sounds retrieved through text based search. Sound discovery is facilitated by use of semantic searches and sound visual representations (spectrograms). Guided by the motivation to create an intuitive tool to support music practice that could suit both novice and trained musicians, we developed and improved the system in a continuous process, gathering frequent feedback from a range of users with various skills. We assessed the prototype with 18 non musician and musician participants during free music improvisation sessions. Results indicate that the system was found easy to use and supports creative collaboration and expressiveness irrespective of musical ability. We identified further design challenges linked to creative identification, control and content quality.

Playsound.space: Inclusive Free Music Improvisations Using Audio Commons.

With the advent of online audio resources and web technologies, digital tools for sound designers and music producers are changing. The Internet provides access to hundreds of thousands of digital audio files, from human- and nature-related environmental sounds, instrument samples and sound effects, to produced songs ready to use in media production. In relation to the vast amount of creative content available online, an emerging community has forged a culture of sharing. Creative Commons (CC) appears as a legal framework to support such initiative enabling the reuse and remix of creative artifacts. Through exploring five use cases, we discuss key concepts and challenges related to the use of Audio Commons content for linear media production.

Leveraging Online Audio Commons Content for Media Production

Smart cities are urban areas with sensor networks that collect data used towards efficient management As a source of ubiquitous data, smart city initiatives present opportunities to enhance inhabitants' urban awareness However, making sense of smart city data is challenging and there is a gap between available data and end-user applications Sonification emerges as a promising method for the interpretation of smart city data and the production of novel musical experiences In this paper, we first present the smart city paradigm We then cover the topics of ubiquitous and mobile music, followed by an overview of sonification research Finally, we propose an approach entitled ubiquitous sonification and present the initial design of a speculative use case for musical smart city systems, leveraging user and urban data to inform behaviour

Musical Smart City: Perspectives on Ubiquitous Sonification.

Soundscape composition is an art form that has grown from acoustic ecology and soundscape studies. Current practices foster a wide range of approaches, from the educational and documentary function of the world soundscape project (WSP) to the creation of imaginary sonic worlds supported by theories of acousmatic and electroacoustic music.Sonification is the process of rendering audio in response to data, and is often used in scenarios where visual representations of data are impractical. The field of auditory display has grown in isolation to soundscape composition, however fosters conceptual similarities in its representation of information in sonic form.This paper investigates the use of data sonification as a narrative tool in soundscape composition. A soundscape has been created using traditional concrete sounds (fixed media recorded sound objects), augmented with sonified real-time elements. An online survey and listening experiment was conducted, which asked participants to rate the soundscape on its ability to communicate specific detail with regard to environmental and social elements contained within. Research data collected shows a strong ability in participants to decode and comprehend additional layers of narrative information communicated through the soundscape.

Jon Pigrem

Papers

Datascaping: Data Sonification as a Narrative Device in Soundscape Composition