Delegating Creativity: Use of Musical Algorithms in Machine Listening and Composition
01 Jan 2014-pp 127-158
TL;DR: This chapter considers the role of creative music systems in music making today and speculate whether this is going to be the next revolution in music cultures and argues that modeling of creative processes in music can be done within a framework of cognitive probabilistic modeling.
Abstract: The affinity between mathematics and music has always spurred musicians to define various aspects of their practice in formal terms. This led historically to important innovations in tuning systems , design of new sounds and advances in music theory as well as emergence of new musical languages and their cultural expressions . Today the technology offers more intelligent and complex ways for automatic manipulation of musical knowledge and structure. This raises new challenges for understanding aspects of music creativity and music perception that have largely remained beyond the reach of formal algorithmic composition and generative music procedures . In this chapter we will consider the role of creative music systems in music making today and speculate whether this is going to be the next revolution in music cultures. Autonomous (music for games), human assisted (meta-creation) and recombinant and audience interactive systems (music apps) will be considered as examples of novel directions in music creation and public engagement with musical contents. It is argued that modeling of creative processes in music can be done within a framework of cognitive probabilistic modeling , laying the foundation for novel research on music information dynamics and action-cognition models applied to music.
Citations
More filters
Journal Article•
TL;DR: This paper is a survey and an analysis of different ways of using deep learning (deep artificial neural networks) to generate musical content, based on the analysis of many existing deep-learning based systems for music generation selected from the relevant literature.
Abstract: This paper is a survey and an analysis of different ways of using deep learning (deep artificial neural networks) to generate musical content.
We propose a methodology based on five dimensions for our analysis:
• Objective
– What musical content is to be generated?
Examples are: melody, polyphony, accompaniment or counterpoint.
– For what destination and for what use?
To be performed by a human(s) (in the case of a musical score), or by a machine (in the case of an audio file).
• Representation
– What are the concepts to be manipulated?
Examples are: waveform, spectrogram, note, chord, meter and beat.
– What format is to be used?
Examples are: MIDI, piano roll or text.
– How will the representation be encoded? Examples are: scalar, one-hot or many-hot.
• Architecture
– What type(s) of deep neural network is (are) to be used?
Examples are: feedforward network, recurrent network, autoencoder or generative adversarial networks.
• Challenges
– What are the limitations and open challenges?
Examples are: variability, interactivity and creativity.
• Strategy
– How do we model and control the process of generation?
Examples are: single-step feedforward, iterative feedforward, sampling or input manipulation.
For each dimension, we conduct a comparative analysis of various models and techniques and we propose some tentative multidimensional typology. This typology is bottom-up, based on the analysis of many existing deep-learning based systems for music generation selected from the relevant literature. These systems are described in this survey and are used to exemplify the various choices of objective, representation, architecture, challenges and strategies. The last part of the paper includes some discussion and some prospects.
This paper is a simplified (weak DRM1) version of the following book [13]: Jean-Pierre Briot, Gaetan Hadjeres and Francois-David Pachet, Deep Learning Techniques for Music Generation, Computational Synthesis and Creative Systems, Springer, 2019. Hardcover ISBN: 978-3-319-70162-2. eBook ISBN: 978-3-319-70163-9. Series ISSN: 2509- 6575.
228 citations
Additional excerpts
..., in [41]....
[...]
10 Jan 2017
TL;DR: This article takes a state-of-the-art music content analysis system and investigates what causes it to achieve exceptionally high performance in a benchmark music audio dataset, and dissects the system to understand its operation, determine its sensitivities and limitations, and predict the kinds of knowledge it could and could not possess about music.
Abstract: Building systems that possess the sensitivity and intelligence to identify and describe high-level attributes in music audio signals continues to be an elusive goal but one that surely has broad and deep implications for a wide variety of applications. Hundreds of articles have so far been published toward this goal, and great progress appears to have been made. Some systems produce remarkable accuracies at recognizing high-level semantic concepts, such as music style, genre, and mood. However, it might be that these numbers do not mean what they seem. In this article, we take a state-of-the-art music content analysis system and investigate what causes it to achieve exceptionally high performance in a benchmark music audio dataset. We dissect the system to understand its operation, determine its sensitivities and limitations, and predict the kinds of knowledge it could and could not possess about music. We perform a series of experiments to illuminate what the system has actually learned to do and to what extent it is performing the intended music listening task. Our results demonstrate how the initial manifestation of music intelligence in this state of the art can be deceptive. Our work provides constructive directions toward developing music content analysis systems that can address the music information and creation needs of real-world users.
15 citations
Cites background from "Delegating Creativity: Use of Music..."
...It is closer to “the musical surface” [Dubnov et al. 2003; Dubnov and Surges 2014]....
[...]
...If the aim of a music content analysis system is to facilitate creative pursuits, such as composing or performing music in particular styles [Dubnov et al. 2003; Dubnov and Surges 2014], then it must operate with characteristics and criteria relevant to the creative needs of users....
[...]
...…Name, Vol. 0, No. 0, Article 0, Publication date: 0. of audio signal processing [Hiller and Isaacson 1959; Cope 1991; Roads 1996; Dubnov et al. 2003; Pachet 2003; 2011; Collins 2010; Argamon et al. 2010; Dubnov and Surges 2014; Eigenfeldt 2012; Eigenfeldt and Pasquier 2013b; Eigenfeldt 2013]....
[...]
TL;DR: The discourse of AI’s inevitability is critique and the ways in which machine learning must frame or reframe cultural and aesthetic practices in order to automate them, in service of digital distribution, recognition, and recommendation infrastructures are shown.
Abstract: This paper traces the infrastructural politics of automated music mastering to reveal how contemporary iterations of artificial intelligence (AI) shape cultural production. The paper examines the e...
9 citations
TL;DR: In this article, Assumptions from interdisciplinary studies on self-reflection were evaluated using Video Interactive VST Orchestra, a music ensemble for autonomous systems without any musical knowledge.
Abstract: Can autonomous systems be musically creative without musical knowledge? Assumptions from interdisciplinary studies on self-reflection are evaluated using Video Interactive VST Orchestra, a ...
4 citations
Cites methods from "Delegating Creativity: Use of Music..."
...Approaches which adopt a dictionary-basedmachine-learningmodel for the imitation of style (Dubnov, Assayag, Lartillot, & Bejerano, 2003; Dubnov & Surges, 2014), can also be capable of imposing stylistic constraints to the generation process (Pachet, 2016)....
[...]
01 Aug 2017
TL;DR: This paper presents an approach for representing space-filling curves by sound, aiming to add a new way of perceiving their beautiful properties, and shows how previous findings on the power spectrum of pitch fluctuations in music suggest that the answer depends on the number of dimensions of the space- filling curve.
Abstract: This paper presents an approach for representing space-filling curves by sound, aiming to add a new way of perceiving their beautiful properties. In contrast to previous approaches, the representation is such that geometric similarity transformations between parts of the curve carry over to auditory similarity transformations between parts of the sound track. This allows us to sonify space-filling curves, in some cases in up to at least five dimensions, in such a way that some of their geometric properties can be heard. The results direct attention to the question whether space-filling curves exhibit a structure that is similar to music. I show how previous findings on the power spectrum of pitch fluctuations in music suggest that the answer depends on the number of dimensions of the space-filling curve.
3 citations
Cites methods from "Delegating Creativity: Use of Music..."
...Bach and became the basis for the twentieth century twelve-tone (dodecaphonic) serial techniques of Arnold Schoenberg” [8]....
[...]
...[7] “Formal manipulations such as retrograde (backward motion) or inversion (inverting the direction of intervals in a melody) are found in the music of J. S. Bach and became the basis for the twentieth century twelve-tone (dodecaphonic) serial techniques of Arnold Schoenberg” [8]....
[...]
References
More filters
TL;DR: The compression ratio achieved by the proposed universal code uniformly approaches the lower bounds on the compression ratios attainable by block-to-variable codes and variable- to-block codes designed to match a completely specified source.
Abstract: A universal algorithm for sequential data compression is presented. Its performance is investigated with respect to a nonprobabilistic model of constrained sources. The compression ratio achieved by the proposed universal code uniformly approaches the lower bounds on the compression ratios attainable by block-to-variable codes and variable-to-block codes designed to match a completely specified source.
5,844 citations
TL;DR: The proposed concept of compressibility is shown to play a role analogous to that of entropy in classical information theory where one deals with probabilistic ensembles of sequences rather than with individual sequences.
Abstract: Compressibility of individual sequences by the class of generalized finite-state information-lossless encoders is investigated. These encoders can operate in a variable-rate mode as well as a fixed-rate one, and they allow for any finite-state scheme of variable-length-to-variable-length coding. For every individual infinite sequence x a quantity \rho(x) is defined, called the compressibility of x , which is shown to be the asymptotically attainable lower bound on the compression ratio that can be achieved for x by any finite-state encoder. This is demonstrated by means of a constructive coding theorem and its converse that, apart from their asymptotic significance, also provide useful performance criteria for finite and practical data-compression tasks. The proposed concept of compressibility is also shown to play a role analogous to that of entropy in classical information theory where one deals with probabilistic ensembles of sequences rather than with individual sequences. While the definition of \rho(x) allows a different machine for each different sequence to be compressed, the constructive coding theorem leads to a universal algorithm that is asymptotically optimal for all sequences.
3,753 citations
Book•
01 Jan 1956
TL;DR: Kraehenbuehl and McAllester as discussed by the authors studied the relationship between pattern and meaning in music, and provided a basis for meaningful discussion of emotion and meaning of all art.
Abstract: "Altogether it is a book that should be required reading for any student of music, be he composer, performer, or theorist. It clears the air of many confused notions . . . and lays the groundwork for exhaustive study of the basic problem of music theory and aesthetics, the relationship between pattern and meaning."-David Kraehenbuehl, "Journal of Music Theory" "This is the best study of its kind to have come to the attention of this reviewer."-Jules Wolffers, "The Christian Science Monitor " "It is not too much to say that his approach provides a basis for the meaningful discussion of emotion and meaning in all art."-David P. McAllester, "American Anthropologist " "A book which should be read by all who want deeper insights into music listening, performing, and composing."-Marcus G. Raskin, "Chicago Review "
2,239 citations
TL;DR: The scientific investigation of behavioral processes by animal learning theory and economic utility theory has produced a theoretical framework that can help to elucidate the neural correlates for reward functions in learning, goal-directed approach behavior, and decision making under uncertainty.
Abstract: The functions of rewards are based primarily on their effects on behavior and are less directly governed by the physics and chemistry of input events as in sensory systems. Therefore, the investigation of neural mechanisms underlying reward functions requires behavioral theories that can conceptualize the different effects of rewards on behavior. The scientific investigation of behavioral processes by animal learning theory and economic utility theory has produced a theoretical framework that can help to elucidate the neural correlates for reward functions in learning, goal-directed approach behavior, and decision making under uncertainty. Individual neurons can be studied in the reward systems of the brain, including dopamine neurons, orbitofrontal cortex, and striatum. The neural activity can be related to basic theoretical terms of reward and uncertainty, such as contiguity, contingency, prediction error, magnitude, probability, expected value, and variance.
1,419 citations
"Delegating Creativity: Use of Music..." refers background or methods in this paper
...The mechanisms studied in the reward systems of the brain include dopamine neurons, orbitofrontal cortex, and striatum, mechanisms that can be related to basic theoretical terms of reward and uncertainty, such as contiguity, contingency, prediction error, magnitude, probability, expected value, and variance (Schultz 2006; Salimpoor et al. 2011)....
[...]
...done by drawing upon animal learning and economic utility theories (Schultz 2006)....
[...]
TL;DR: A formal Bayesian definition of surprise is proposed to capture subjective aspects of sensory information and it is shown that Bayesian surprise is a strong attractor of human attention, with 72% of all gaze shifts directed towards locations more surprising than the average.
1,407 citations