Emotion and Meaning in Music

This paper is a survey and an analysis of different ways of using deep learning (deep artificial neural networks) to generate musical content.

We propose a methodology based on five dimensions for our analysis:
• Objective
– What musical content is to be generated?
Examples are: melody, polyphony, accompaniment or counterpoint.
– For what destination and for what use?
To be performed by a human(s) (in the case of a musical score), or by a machine (in the case of an audio file).
• Representation
– What are the concepts to be manipulated?
Examples are: waveform, spectrogram, note, chord, meter and beat.
– What format is to be used?
Examples are: MIDI, piano roll or text.
– How will the representation be encoded? Examples are: scalar, one-hot or many-hot.
• Architecture
– What type(s) of deep neural network is (are) to be used?
Examples are: feedforward network, recurrent network, autoencoder or generative adversarial networks.
• Challenges
– What are the limitations and open challenges?
Examples are: variability, interactivity and creativity.
• Strategy
– How do we model and control the process of generation?
Examples are: single-step feedforward, iterative feedforward, sampling or input manipulation.

For each dimension, we conduct a comparative analysis of various models and techniques and we propose some tentative multidimensional typology. This typology is bottom-up, based on the analysis of many existing deep-learning based systems for music generation selected from the relevant literature. These systems are described in this survey and are used to exemplify the various choices of objective, representation, architecture, challenges and strategies. The last part of the paper includes some discussion and some prospects.

This paper is a simplified (weak DRM1) version of the following book [13]: Jean-Pierre Briot, Gaetan Hadjeres and Francois-David Pachet, Deep Learning Techniques for Music Generation, Computational Synthesis and Creative Systems, Springer, 2019. Hardcover ISBN: 978-3-319-70162-2. eBook ISBN: 978-3-319-70163-9. Series ISSN: 2509- 6575.

Deep Learning Techniques for Music Generation - A Survey

In the paper, we chronicle the instantiation and adventures of the Stanford Laptop Orchestra (SLOrk), an ensemble of laptops, humans, hemispherical speaker arrays, interfaces, and, more recently, mobile smart phones. Motivated to deeply explore computer-mediated live performance, SLOrk provides a platform for research, instrument design, sound design, new paradigms for composition, and performance. It also offers a unique classroom combining music, technology, and live performance. Founded in 2008, SLOrk was built from the ground-up by faculty and students at Stanford University's Center for Computer Research in Music and Acoustics (CCRMA). This document describes 1) how SLOrk was built, 2) its initial performances, and 3) the Stanford Laptop Orchestra as a classroom. We chronicle its present, and look to its future.

Stanford laptop orchestra (slork)

Musical agents are artificial agents that tackle musical creative tasks, partially or completely This review of musical agents combines the terminology of Generative Arts (artistic practice) and t

Musical agents: A typology and state of the art towards Musical Metacreation

In this work the problem of guided improvisation is approached and elaborated; then a new method, Variable Markov Oracle, for guided music synthesis is proposed as the first step to tackle the guided improvisation problem. Variable Markov Oracle is based on previous results from Audio Oracle, which is a fast indexing and recombination method of repeating sub-clips in an audio signal. The newly proposed Variable Markov Oracle is capable of identifying inherent datapoint clusters in an audio signal while tracking the sequential relations among clusters at the same time. With a target audio signal indexed by Variable Markov Oracle, a query-matching algorithm is devised to synthesize new music materials by recombination of the target audio matched to a query audio. This approach makes the query-matching algorithm a solution to the guided music synthesis problem. The query-matching algorithm is efficient and intelligent since it follows the inherent clusters discovered by Variable Markov Oracle, creating a query-by-content result which allows numerous applications in concatenative synthesis, machine improvisation and interactive music system. Examples of using Variable Markov Oracle to synthesize new musical materials based on given music signals in the style of Jazz are shown.

Guided Music Synthesis with Variable Markov Oracle.

A system is described which uses the Audio Oracle algorithm for music analysis and machine improvisation. Some improvements on previous Factor Oracle-based systems are presented, including automatic model calibration based on measures from Music Information Dynamics, facilities for compositional structuring and automation, and an audio-based query mode which uses the input signal to influence the output of the generative system.

Feature Selection and Composition Using PyOracle

The affinity between mathematics and music has always spurred musicians to define various aspects of their practice in formal terms. This led historically to important innovations in tuning systems , design of new sounds and advances in music theory as well as emergence of new musical languages and their cultural expressions . Today the technology offers more intelligent and complex ways for automatic manipulation of musical knowledge and structure. This raises new challenges for understanding aspects of music creativity and music perception that have largely remained beyond the reach of formal algorithmic composition and generative music procedures . In this chapter we will consider the role of creative music systems in music making today and speculate whether this is going to be the next revolution in music cultures. Autonomous (music for games), human assisted (meta-creation) and recombinant and audience interactive systems (music apps) will be considered as examples of novel directions in music creation and public engagement with musical contents. It is argued that modeling of creative processes in music can be done within a framework of cognitive probabilistic modeling , laying the foundation for novel research on music information dynamics and action-cognition models applied to music.

Delegating Creativity: Use of Musical Algorithms in Machine Listening and Composition

NRCI is a suite of Pd tools designed to facilitate laptop ensemble performance, and to foster a culture of group music-making which emphasizes custom software design and improvisational practice. NRCI provides a large library of abstractions with a consistent interface; these modules ease the challenge of instrument design for novices, and aid in rapid development and live coding for advanced users. NRCI also implements a network protocol (using OSC) which provides for the wireless exchange of control data and messaging in both improvisational and pre-structured situations. NRCI is designed for and tested by MiLO (the Milwaukee Laptop Orchestra), and is freely available online.

/pdf/nrci-software-tools-for-laptop-ensemble-3p500vapz6.pdf

Nrci: software tools for laptop ensemble

This article describes the use of second-order all-pass filters as components in a feedback network, with parameters made time varying to enable effects such as phase distortion in a generative audio system. The term "audio" is used here to distinguish from generative "music" systems, emphasizing the strong coupling between processes governing the production of high-level music and lower-level audio. The classical time-invariant implementation of an all-pass filter is subject to issues of instability that can arise when time-invariant filter parameters are allowed to vary over time. These instabilities are examined, along with the adoption of a power-preserving rotation matrix formulation of the all-pass filter to ensure stability and ultimately an improved synthesis for a generative audio system.

Generative audio systems using power-preserving all-pass filters

This work builds upon existing work in which second-order allpass filters are used in a feedback network, with parameters made time-varying to enable effects such as phase distortion in a generative audio system; the term “audio” is used here to distinguish from generative “music” systems, emphasizing the strong coupling between processes governing the production of high-level music and lower-level audio. The previous system was subject to issues of instability that can arise when time-invariant filter parameters are allowed to vary over time. These instabilities are examined herein, along with the adoption of a power-preserving rotation matrix formulation of the allpass filter to ensure stability and ultimately an improved synthesis for a generative audio system.

Greg Surges

Papers

Feature Selection and Composition Using PyOracle

Delegating Creativity: Use of Musical Algorithms in Machine Listening and Composition

Nrci: software tools for laptop ensemble

Generative audio systems using power-preserving all-pass filters

Generative Feedback Networks Using Time-Varying Allpass Filters