This is a post-print (author’s final draft) of an article in the e-journal
Axiomathes (2005), 15, pp. 229-266 by Springer. [Original page numbers
between square brackets]. Details of the definitive version are available at
http://www.springerlink.com/content/h1j0776t0552wu6x/
MARK REYBROUCK
Running head: Music cognition and event perception
Article type: Research article
Title: A biosemiotic and ecological approach to music cognition: event
perception between auditory listening and cognitive economy
Author: Mark Reybrouck
Affiliation: Catholic University of Leuven, Section of musicology
Correspondence address: Eikendreef 21, B-8490 Varsenare, Belgium
Tel.(++32)(50)380277
Fax(++32)(16)324872
E-mail: Mark.Reybrouck@arts.kuleuven.be
MARK REYBROUCK
A BIOSEMIOTIC AND ECOLOGICAL APPROACH TO MUSIC
COGNITION: EVENT PERCEPTION BETWEEN AUDITORY
LISTENING AND COGNITIVE ECONOMY
ABSTRACT. This paper addresses the question whether we can conceive of
music cognition in ecosemiotic terms. It claims that music knowledge must be
generated as a tool for adaptation to the sonic world and calls forth a shift from a
structural description of music as an artifact to a process-like approach to
dealing with music. As listeners, we are observers who construct and organize
our knowledge and bring with us our observational tools. What matters is not
merely the sonic world in its objective qualities, but the world as perceived. In
order to make these claims operational we can rely on the ecological concept of
coping with the sonic world and the cybernetic concepts of artificial and
adaptive devices. Listeners, on this view, are able to change their semantic
relations with the sonic world through functional adaptations at the level of
sensing, acting and coordinating between action and perception. This allows us
to understand music in functional terms of what it affords to us and not merely
in terms of its acoustic qualities. There are, however, degrees of freedom and
constraints which shape the semiotization of the sonic world. As such we must
consider the role of event perception and cognitive economy: listeners do not
perceive the acoustical environment in terms of phenomenological descriptions
but as ecological events.
KEY WORDS: adaptation and adaptive control, cognitive economy, coping
behavior, ecosemiotics, enactive and experiential cognition, epistemic control
system,
event perception, listener–environment interaction, musical epistemology
1. INTRODUCTION: FROM STRUCTURAL DESCRIPTION TO COPING
WITH THE SONIC WORLD
[229] This paper is about musical epistemology. It addresses the question
whether we can conceive of music cognition in biosemiotic and ecological
terms. The question is tedious, as music processing is a skilled activity which is
dependent on several higher functions of the brain. It embraces activities as
different as listening, performing and even composing or improvising and offers
an unrivaled opportunity to investigate the neural correlates of skill acquisition
besides unique abilities as the recognition of absolute pitch or musical sight-
reading. [230] There is a vast body of literature on the effects of music
performance as a skilled activity which requires the simultaneous integration of
multimodal sensory and motor information with multimodal sensory feedback
mechanisms to monitor performance (Gaser and Schlaug 2003). Several
behavioral, neurophysiological and neuroimaging studies have explored the
highly specialized sensorimotor, auditory, visual-spatial, auditory-spatial and
memory skills of musicians while performing motor, auditory and
somatosensory tasks, and the search for ‘‘anatomical markers’’ of these
extraordinary skills has resulted in findings which are at least fascinating: there
are ‘‘functional’’ and ‘‘structural’’ differences between the brains of musicians
and non-musicians which are illustrative of the neural plasticity and structural
adaptation of brain tissue in response to intense environmental demands during
critical periods of brain maturation (Gaser and Schlaug 2003). Music
processing, further, is also related to language development, human
communication, brain development and evolution (Besson and Schon 2001;
Gray et al. 2001). It is legitimate, therefore, to consider the biological bases of
dealing with music.
Music, as a ‘‘man-made construction’’ or ‘‘artifact’’, however, is biologically
non-relevant: there is no causal relation between the music as a stimulus and any
direct reaction to this stimulus. Music as a ‘‘physical stimulus’’ or ‘‘sound’’, on
the contrary, is able to elicit reactions which can be explained in physiological
and biological terms (Reybrouck 2001a; Martinelli 2002; see also the extensive
literatureon music therapy and biosemiotics). The reactions can be either direct
or cognitively mediated, with a gradual rather than a qualitative distinction
between the processing of ‘‘sound’’ and ‘‘music’’. It allows us to conceive of
music as part of the sonic environment and of listening as a way of coping with
this environment. Listening, on this view, relies on music knowledge that must
be generated as a tool for adaptation to the sonic world and that involves
listening strategies which are the outcome of interactions between the listener as
an organism and the music as environment. This ‘‘interactional’’ approach is a
core assumption of the ecological approach to cognition, and is somewhat
opposed to the scientific persuasion that the world can be described in a
language that is incommensurable with our experiences. As such, it entails a
transition from a structural description of the music to a process-like approach
to coping with the sonic world.
[231] Arguing on these lines, we can conceive of the process of listening as
establishing ‘‘semiotic relations’’ between an organism and its environment.
This is an ecosemiotic position (for a critical definition, see Kull 1998b) with
the listener – as an organismus semioticus (Nöth 1998) – showing adaptive
behavior in his or her interactions with the music as environment (Reybrouck
2001c). The example of a musician who plays a violin is illustrative at his point:
in order to produce a beautiful sound he relies on sensory–motor integration,
‘‘shaping’’ the sound through the perception of the sounding result which must
match the internalized representation of this sound. Sound production, on this
view, entails the reciprocity of ‘‘doing’’ and ‘‘undergoing’’,
as stressed already in the pragmatic philosophy of Dewey:
In short, art, in its form, unites the very same relation of doing and undergoing,
outgoing and incoming energy, that makes an experience to be an experience … The
doing or making is artistic when the perceived result is of such a nature that its
qualities as perceived have controlled the question of production. … The artist
embodies in himself the attitude of the perceiver while he works. (1934, p. 48)
The reciprocity of doing and undergoing is typical of sensory–motor integration.
This is obvious in ‘‘playing’’ music, but it applies also to the process of
‘‘listening’’, with the listener imagining or simulating mentally the manifest
movements of the players (see Delalande 1984; Lidov 1987). There is, in fact,
empirical evidence that motor imagery and motor execution involve activities of
very similar cerebral motor structures at all stages of motor control (Crammond
1997). It allows us to stress the continuity between ‘‘sensory–motor
integration’’ an ‘‘ideomotor simulation’’, the former dealing with movements
that are actually executed in real-time, the latter with movements that are
simulated at an ideational level of motor imagery (Paillard 1994b; Reybrouck
2001b).
2. FROM CYBERNETICS TO SEMIOTICS
To conceive of music cognition in terms of coping behavior is an
epistemological position that deals with music in ‘‘behavioral’’ terms. Several
options are possible here – playing, listening, composing, improvising – but the
focus of this paper is on the process of listening as a kind of interaction with the
sonic world. This calls forth a processlike approach to dealing with music which
can be described in terms of ‘‘experiential’’ and ‘‘enactive’’ cognition – both
terms will be explained throughout this paper – which, in turn, can be described
[232] in operational terms that rely on the concept of control system (see Figure
1). It embraces the four major elements of adaptive control – input, output,
central processing and feedback – and provides a common language for the
description of adaptive behavior in general, leaning heavily on the conceptual
work of cybernetics and artificial devices.
Figure 1. The basic schema of a control system with the four major moments of adaptive
control.
2.1. The control system as a starting point
Cybernetics is a unifying discipline which brings together concepts as different
as the flow of information, control by feedback, adaptation, learning and self-
organization (see Bateson 1973; Brier 1999a, b; Cariani 2001a). It allows us to
conceive of devices which behave as ‘‘autonomous agents’’ which, according to
Emmeche [are not only] input-output devices, but [which] move around as
cybernetic systems by their own motor modules guided by sensors, making
decisions, having the capacity of acting more or less intelligently given only
partial information, learning by their mistakes, adapting to heterogeneous and
changing environments, and having a sort of life of their own. (2001, p. 659)
A central metaphor of cybernetics is the cyclic image of brain and environment,
with internal sets of feedback loops themselves having feedback connections to
the environment and being completed through them (McCulloch 1989; Cariani
2001a, b). It is represented in the basic schema of the control system, which is
an interesting conceptual tool that allows us to conceive of music users as
‘‘devices’’.
2.2. The listener as an adaptive device
Music users can be conceived of as adaptive devices that are able to change their
semantic relations with the world (Reybrouck 2001c; and for a general
discussion of adaptive devices, see Rosen 1978;
[233] Pattee 1982, 1985; Cariani 1991, 1998a). They can arbitrarily choose what
kinds of ‘‘distinctions’’ are to be made (perceptual categories,features and
primitives), what kinds of ‘‘actions’’ are done on the environment (primitive
action categories), and what kinds of ‘‘coordinative mappings’’ are carried out
between their actions and distinctions. As such, they behave as adaptive
systems, which, according to Cariani (2001b), acquire a degree of ‘‘epistemic
autonomy’’:
… adaptive systems …continually modify their internal structure in response to
experience. To the extent that an adaptive epistemic system constructs itself and
determines the nature of its own informational transactions with its environs, that
system achieves a degree of epistemic autonomy relative to its surrounds. (2001b, p.
60)
Adaptive devices can change the informational relationships with their
environment through altering the basic functions of sensing, coordinating and
acting. The ‘‘sensing function’’ can be changed through modifying or
augmenting the sensors. It allows the device to choose its own perceptual
categories and to control the types of empirical information it can access
through the basic mechanisms of altering existing sensing functions or adding
additional ones.
According to Cariani (1991, 1998b) there are basically four of them: (i)
prosthesis or adaptive fabrication of new front-ends for existing sensors, (ii)
active sensing or using motor actions to alter what is sensed through interaction
(poking, pushing, bending), (iii) sensory evolution or adaptive construction of
entirely new sensors and (iv) internalized sensing by creating internal, analog
representations of the world out of which internal sensors extract newly relevant
properties (perceptual learning) (1998b, p. 718). The ‘‘effector function’’, on the
other hand, can be modified as well. The device can change its effectors or
coordinate its actions with the sensing and coordinating function as in
‘‘active measurement’’. This is a process of acting on the world and sensing
![Figure 2. The functional cycle after von Uexküll (1957[1934]).](/figures/figure-2-the-functional-cycle-after-von-uexkull-1957-1934-38ulixsr.webp)
