Résumés
Abstract
This paper deals with musical sense-making in a real-time listening situation. Revolving around the ecological conception of organism-environment interaction, it elaborates on the interactions between the listener as an organism and the music as environment. The listener, in this view, can be described in terms of coping behavior that is shaped by biological and ecological constraints. Relying on the seminal work by von Uexküll and Gibson in the fields of biosemiotics and ecology, with a special emphasis on the concepts of functional tone and affordance, listeners are defined as organisms that actively seek for information by carrying out physical and epistemic interactions on the sonic environment. As such, they construct an inner model of the sonic world as the sum total of subjective meanings that are assigned to those elements that receive semantic weight. By stressing the role of functional significance and interactions, this approach is on a continuum with the biosemiotic claims that music knowledge must be generated as a tool for adaptation to the sonic world. Musical sense-making, in this view, relies on several levels of processing, going from low-level reactivity to higher-level processing by the brain.
Keywords:
- Musical Sense-Making,
- Biosemiotics,
- Ecological Perception,
- Musical Affordances,
- Coping Behavior,
- Event Perception
Résumé
Cet article traite de la signification musicale dans une situation d’écoute en temps réel. Articulé autour de la conception écologique de l’interaction organisme-environnement, il explore les interactions entre l’auditeur en tant qu’organisme et la musique en tant qu’environnement. Selon ce point de vue, l’auditeur peut être décrit par son comportement d’adaptation, comportement façonné par des contraintes biologiques et écologiques. S’appuyant sur les travaux précurseurs de von Uexküll et de Gibson dans les domaines de la biosémiotique et de l’écologie, et en mettant particulièrement l’accent sur les concepts de tonalité fonctionnelle et d’affordance, les auditeurs sont définis comme des organismes qui recherchent activement des informations en effectuant des interactions physiques et épistémiques sur l’environnement sonore. Les auditeurs construisent ainsi un modèle intérieur du monde sonore comme la somme totale des significations subjectives attribuées aux éléments qui reçoivent un poids sémantique. En soulignant le rôle de la signification fonctionnelle et des interactions, cette approche s’inscrit dans une tradition biosémiotique selon laquelle la connaissance de la musique doit être générée en tant qu’outil d’adaptation au monde sonore. Dans cette perspective, la création de sens musical repose sur plusieurs niveaux de traitement, de la réactivité de bas niveau jusqu’aux traitements cérébraux plus élaborés.
Mots-clés :
- Création de sens musical,
- biosémiotique,
- perception écologique,
- affordances musicales,
- comportement d’adaption,
- perception d’événements
Corps de l’article
Introduction
Music can be considered as a vibrational event that impinges upon the body and the mind. As such, it can be defined at two levels of description : the physical level of the sounds and the physiological level of reactions to the sounds. The former calls forth the acoustics of sound production and propagation; the latter is related to sensation and perception. Both levels involve each other, but the search for linear causal relations between them has been difficult and unconvincing up to now. There is, in fact, no “pharmaceutical model” that explains the effects of music in terms of structural features of the sounds (Sloboda 2005 : 319). The effects, moreover, are not only triggered by human physiology, but are mediated also by the listener’s choices and mental states. It makes sense, therefore, to conceive of musical sense-making in interactional and enactive terms as a kind of active perceptual exploration with a lot of freedom for each individual listener. There is, however, the level of psychophysics and to some extent also the level of psychobiology where some linearity between the sensory input and the corresponding reactions by the listener can be shown. There are, in fact, some triggering forces in the physical features of the sound that can modulate perceptual reactions in a quasi-causal way (Reybrouck & Eerola 2017a). It has been demonstrated, e.g., that there are several categories of psychophysical problems in general that seem to be invariant among humans, such as the determination of absolute and differential thresholds, the judgment of equality, order, equality of intervals on a scale, equality of ratios and stimulus rating (Stevens 1951 : 33).
Translated to the domain of music, this means that we should look for some reliable correlations between acoustic signals and their perceptual processing, such as, e.g., the relationship between a physical variable like intensity and a perceptual variable like perceived loudness. In music studies, however, the term psychophysical has been used also in a broader sense to designate those characteristics to which basic auditory processes naturally respond. They encompass properties such as tempo, pitch range, melodic complexity and rhythmic complexity, all of which can be defined and assessed independently of musical conventions of any particular culture (Balkwill 1997). The level of psychobiology, on the other hand, deals with the transition from physical or acoustic stimuli to mental experiences (Uttal 1973). It is concerned with the mind-body relationship, in stating that psychological properties arise out of physiological processes. As such, it should be possible to provide explanations for psychological functions by observing the underlying physiological processes, which is, in a nutshell the axiom of psychobiological equivalence (Uttal 1978 : 10; Reybrouck 2013).
The psychobiological claims are challenging. They provide empirical grounding for insights that were intuitive to some extent. Relying on results from neuroimaging and morphometric studies, they revolve around three major domains : (i) the localization of functions in the brain and the search for anatomical markers for musical skills; (ii) the representation or coding of cognitive processes in neural networks, and (iii) the dynamic change or learning as a result of experience. The latter has furthered a considerable number of studies on neural plasticity and structural and functional adaptation of the brain as the result of continuous and prolonged musical experiences (Reybrouck & Brattico 2016).
Dealing with Music : Broadening of Scope
There is a subjective element in musical sense-making that goes beyond the level of physical-sensorial description of the sounds and the physiological processing by the listener. Subjectivity, however, is not to be equated with unlimited freedom. There are, in fact, biological and ecological constraints on what is considered as music and on the ways of coping with the sounds. It makes sense, therefore, to elaborate on the concept of music and listener in ecological terms as an interaction between an organism and its environment.
Defining music, first, is not an easy task, as music is not a well-defined and static category but an umbrella term that encompasses both Western art music, popular music and the musics of the world. There is even a whole universe of sounds and noises that can be qualified also as music, dependent on the deliberate intentions of the listener who can raise the sonic environment to the level of music (Reybrouck 2015). This holds true for sounds in general but also for noises and all kinds of noise music (Cassidy & Einbond 2013; Cumming 2000; Voegeli 2010). It makes sense, therefore, to conceive of music not merely in terms of historic or geographic styles or genres, but in terms of sound. Music, in that view, is a vibrational event that impinges upon the senses (Eidsheim 2012), stressing the materiality of music-as-heard with sound as its major defining category. It makes it possible to conceive of music as a subclass of the more encompassing sonic universe and to cope with music in terms of a sounding environment.
Listeners, on the other hand, can “cope” with this environment. They can be considered as organisms that interact with their environment, to use the ecological framework that stresses the reciprocity of perceiver and environment (Lombardo 1987; Reybrouck 2005). The question, however, what these interactions are. There is, in fact, a distinction between physical actions – as in playing a music instrument – and epistemic interactions with the sounds. The latter, especially, are interesting as they break away from biological constraints, which are related to the use of sensory and motor tools as natural interfaces of the body to interact with its environment. They make it possible to introduce cognitive tools as well and to enlarge the scope of interactions that occur between listeners and the sounding music.
Musical sense-making, in this view, is an active process of knowledge construction (Reybrouck 2017a), with a shift from naïve realism that takes the world as it is to cognitive realism that sees the world as having the mark of our own structure. Knowledge, accordingly, is to be defined as the result of an ongoing interpretation that emerges from our capacities of understanding which are rooted in the structure of our biological embodiment (Varela et al. 1991). Such a view is typically a non-objectivist orientation to semantics : it views cognition as enaction and is consonant with the “experiential” approach to cognition, which accounts for what meaning is to human beings, rather than trying to replace it by reference to a metaphysical account of a reality external to human experience (Lakoff 1987).
Music as Environment
Conceiving of music as environment entails an ecological approach to listening (Reybrouck, 2015). Rather than equating it in terms of musical works, it espouses a broader definition of music as a subuniverse of the more encompassing sonic universe, which can be considered as the totality of sounding elements that represent all possible combinations of individual vibrational events (Cogan 1984). Some of them are able to elicit reactions which can be explained in physiological and biological terms and which can be processed in a rather direct way. Other elements, on the contrary, need more processing efforts and are meaningful only to the extent that they are cognitively mediated. There is, as such, a multiplicity of levels of processing with a continuum from low level reactivity to higher level cognitive processing of the sounds (Reybrouck & Eerola 2017).
Some of the sounding stimuli are biologically relevant – as in the case of a threatening environment – but most of them are not. In both cases, however, listeners may be invited to “cope” with them either in terms of approach or withdrawal. This is obvious in an ecological setting, such as living in a challenging environment, where survival depends on avoiding predators as well as on finding food and the construction of a shelter. It is possible, however, to apply this also to the realm of music with a gradual rather than a qualitative distinction between the processing of environmental sounds and music. All sounds, in fact, are caused by vibration, but not all vibrational events are perceived in a similar way. There is, first, the much-used distinction between sound and noise. Sounds, in a musical context, are equated mainly with periodic and complex vibrations that are generated by musical instruments and/or by the human voice. They produce harmonics – vibration frequencies that add up to the fundamental vibration frequency and that multiples of these frequencies –, which are perceived mostly as pleasant and preferred auditory stimuli. Noise, on the contrary, consists of vibrations that result in irregular frequencies, with inconsistencies of tension, stress and configuration, and which mostly produces fatigue, stress, hyperalerting responses and startle reflexes in listeners (Maschke et al. 2000; Standley 2002). This distinction, however, is somewhat arbitrary as there are cases of noise music that are qualified without contest as music. Many musical sounds, moreover, do not consist merely of harmonic sounds but encompass also a lot of noisy elements, due to the mechanics of sound production. Yet, there is a conception of ambient noise, that can be defined as the totality of noises that can be heard in one’s environment and that are present but not chosen by the listener (Wagner 1994). As such, they exist in the environment but without the possibility of controlling their volume, duration, location, or cause/effect relationships.
There is, further, a lot of freedom in the way how listeners direct their attention to the sonic universe. There are, however, biological and ecological constraints on coping with the sounds. The former relate to limitations of the sensory modalities (Reybrouck 2017b), such as their sensory range, acuity and resolving power (absolute and differential thresholds). The latter capitalize on the former but go beyond the level of sensation and psychophysical processing. Rather than dealing with the sonic environment in terms of its acoustic properties, listeners are attuned to perceive sounds as auditory events (Gaver 1993). This is in consonance with the claims of event perception in general (Hommel et al. 2001; Wittmann 2011) with events being defined as higher-order variables that are characterized by an extension in time. It means, further, that listeners have access to information in a rather direct way, relying on “pickup” rather than on “processing” of the sounds. As such, they search for ecological events, which are continuous in their unfolding but discrete in their labeling and which make them apt for recognition and identification without much cognitive effort (Balzano 1986; Lombardo 1987).
Such “direct perception” is grounded in ecological theory, which claims that perception occurs immediately without the mind intervening in this process. It involves direct contact with the sensory stimuli with estimation of the content occurring in a lock-and-key approach (Michaels & Carello 1981). It is much less demanding, in fact, to recognize a sounding object or event as a discrete entity with propositional character than to experience its sonorous articulation through time. Yet, information pickup, is not merely a passive phenomenon but an active search for information as mapped out already by Gibson who defined perceivers as perceptual systems that “search out” information which becomes “obtained” information by picking up information which is already structured and ordered as part of an organism-environment ecosystem (Gibson 1966 : 47). Perceivers, in this view, are “tuned” to the information that is considered to be useful : hence the role of key concepts as attunement, reciprocity and resonance and the corresponding perceptual processes of detection, discrimination, recognition and identification (Gibson 1966, 1979; see also Reybrouck 2001a, 2005).
Event Perception and the Detection of Sonic Invariants
The perception of events has adaptive value. By providing cognitive schemes, which are helpful in making sense of the environmental world, they are helpful in schematizing the physical structures in the sonic environment. It is up to the listener, however, to decide to what extent these structures are considered relevant to their adaptive efforts. Event perception, in this view, can be considered as a schematizing process that “ecologizes” the stuff of the world either to render it more assailable by the organisms or to accommodate the organism to its environment (Shaw & Hazelett 1986). It entails categorical rather than auditory perception to the extent that it relies on discrete processing in which the event is heard directly as a global event rather than a sounding articulation over time. It is a conception which is related directly to two basic principles of categorization, namely cognitive economy and the principle of reality.
As to the first, it is a major aim of categorization to provide the maximum of information with the least cognitive effort – this is the principle of cognitive economy –, by responding to them in terms of their class membership rather than their uniqueness (Bruner et al. 1956). This means that genuinely diverse inputs lead to a single output, without preserving the shape, size, position and other formal characteristics of the stimulus (Neisser 1987). Categorization, further, mostly starts from the assumption of an implicit ontological realism – as advocated in the early work of Rosch on categorization (Rosch & Lloyd 1978; see also Dubois 1991) –, which means that the perceived world is not unstructured but consists of real and natural discontinuities and co-occurrent properties. This is the principle of reality.
It is easy to translate this to the realm of music and to conceive of “event perception” as a kind of top-down processing of the music with schemata or labels that are assigned to segments of the sonorous unfolding through time. Musical events, in this view, can be defined as higher-order variables which can be described as having time-varying complex acoustic properties with temporal constraints. Perceptual units in the range of 2 to 3 seconds have been identified as allowing event identification over time (Wittmann & Pöppel 1999-2000). Most musical events, further, have a clearly defined time of beginning and ending and have a gross temporal patterning as well (Handel, 1989). They hold a position between “invariance” and “change” with the extraction of invariants pertaining to either static or dynamic features of the stimuli. As such, it is possible to draw a distinction between structural and transformational invariants (Shaw et al. 1996; Michaels & Carello 1981) : structural invariants refer to features that are not or only slowly changing, transformational invariants, on the contrary, refer to styles of change (Shaw & Pittenger 1978). The combined perception of both can be defined in intuitive terms as “something happening to something”, with the “something happening” referring to the transformational and the “something” to the structural invariants (Michaels & Carello 1981 : 26). Recognizing the sound of a string instrument, for example, is a structural invariant, recognizing this sound as being bowed or plucked, is a transformational invariant.
The concepts of invariants and events are tightly intertwined. The invariant act as a kind of glue that unitizes sequences of stimulus information into coherent events, which are extended in time (Bartlett 1984). The latter can be described in terms of their invariants and behave as the basic building blocks which function as units in perception in memory. There is, however, no a priori agreement on their definition. It is possible, for example, to conceive of them at the level of individual notes, as mainly distinguishable discrete distinctions that unfold over time, but notes are not the only audible things that function as events. Especially in music, there is a fuzzy transition between actual sensation and representation. Real perception is limited to a small temporal window – coined as psychical present (Stern 1897) and spacious present (James1890 – through which listeners keep track with the unfolding over time. It can be defined as a “now” with a horizon of retension of what just passed and a protension on what is coming next. It is possible, however, to extend this temporal window to larger structures such as a motif or phrase, or even other temporal extensions. The challenge, however, is to determine the critical transition between an ecological event and a temporal structure that is perceived as something with unit-character, and it can even be questioned whether this distinction is qualitative rather than gradual.
The definition of a temporal gestalt unit is an example that conceives of events as distinct spans of time which are both internally cohesive and externally segregated from comparable time-spans that immediately precede and follow them (Tenney & Polansky 1980). It is somewhat related to the description of an auditory image as a psychological representation of a sound entity which exhibits a coherence in its acoustic behavior (McAdams 1984). Other approaches have a more pronounced semiotic mark in their search for temporal semiotic units, which could be considered as minimal meaningful units for music that convey meaning through their dynamic organization over time (Frey et al. 2014). Starting from an attempt to identify the specific temporal forms that show a development between two and several seconds, a taxonomy of several categories has been identified that could be described in six larger families with the following characteristics : sound parameters remaining constant, evolving monotonously over time, evolving exponentially, first growing and then decreasing, following a Dirac function, or having a specific form (Bootz & Hautbois 2007).
Ecology, Biosemiotics and the Enactive Approach
The ecological approach to listening is a promising area of research. It takes as starting point the dynamic relationship between an organism and its environment, as coined already by Haeckel who conceived of ecology as the science of the relations between the organisms and the environmental outer world (Haeckel 1988/1866). Translated to the realm of music, this should mean that we substitute the listener for the organism and music for the environment (Reybrouck 2001a, 2005).
Such an interactional approach is somewhat at odds with the long-held tradition in science to describe the world in a language that is incommensurable with our experiences. It entails a transition from a structural description of the music – in terms of disembodied categorical terms – to a process-like approach to coping with the sonic world, somewhat related to the distinction between the logogenic and musogenic approach to musical sense-making (Tagg 2013). The logogenic approach assumes that musical meaning is conducive to verbal expression (words), the musogenic approach highlights those properties that can be put adequately in music Or put in other terms : music-structural knowledge can be equated with pre-existing concepts and labels that may be assigned to the sonorous unfolding (logogenic) or with a moment-to-moment experience of the particularities of the musical unfolding in real time (Reybrouck 2017a).
Adherence to the logogenic view has for long been the dominant paradigm in research on musical sense-making with a major focus on the lexico-semantic dimensions of conceptualization, but bypassing largely the particularities of a musical experience (Schiavio et al., 2016). As such, alternative embodied/enactive models of mind – such as the “4E” model of cognition (embodied, embedded, enactive and extended, see Menary 2010) – have challenged this narrow, disembodied approach by defining meaning–making as an ongoing process of dynamic interactivity between an organism and its environment (Barrett 2011; Maiese 2011; Hutto & Myin 2013). Relying on the concept of enactivism as a cross-disciplinary perspective on human cognition that integrates insights from phenomenology and philosophy of mind, cognitive neuroscience, theoretical biology, and developmental and social psychology (Varela et al. 1991; Thompson 2007; Stewart et al. 2010), enactive models understand cognition as embodied and perceptually guided activity that is constituted by circular interactions between an organism and its environment. Through continuous sensorimotor loops (defined by real-time perception-action cycles), the living organism – including the music listener/performer – enacts or brings forth his/her own domain of meaning (Colombetti & Thompson 2008; Reybrouck 2005; Thompson 2005) without separation between the cognitive states of the organism, its physiology, and the environment in which it is embedded. Cognition and mind, in this view, originate in a continuous interplay between an organism and its environment as an evolving dynamic system (Hurley 1998).
The enactive claims bring together the ecological and experiential approach to sense-making. Relying on the body with its sensory and motor tools as the interface for interaction with the environmental sounding world, they sound as a faint echo of the biosemiotic approach that was advocated already in the theoretical writings of von Uexküll. In emphasizing the construction of an inner world as the outcome of interactions with the world, he introduced the concept of functional cycle, as a conceptual tool to describe behavior in terms of perception and action on the basis of a simple recursive loop that provides a description in terms of sensory-motor integration (von Uexküll 1957 [1934]). Contrary to the linearity of a stimulus-reaction chain – as a kind of reactivity to an external environment – he introduced the concept of “circularity” with both receptor and effector cues influencing each other. Every stimulus, in this view, presupposes a readiness to react, but it is up to the perceiver to select the stimuli which are approached as part of his/her subjectively perceived environment or Umwelt, as he coined the term. Such an Umwelt is a private subset of the world at large. It encompasses all the meanings of the world for a particular organism. The main emphasis, in this view, is the construction of an internal model of the world, which is the result of a mapping relations between an organism and its environment. They bring together the world of sensing and acting through processes of signification which invest the objects or events with perceptual and effector tones. As such, functional cycles describe sensory-motor interactions, which can be equated with elementary loops of functioning which consist of sensors, sensory processing, a world model, a commands generator, actuators and the world where changes happen (Meystel 1998).
Extending on this view, it is possible to conceive of listeners as devices with an internal model of the music as environment. It is an idea that has been developed in the domains of cybernetics, robotics and biosemiotics, and which has received new momentum in recent research on adaptive control of percept-action loops in artificial devices (Ziemke & Sharkey 2001) and sensory-motor association learning as a central mechanism that underlies the development of internal models of knowledge (Burianová et al., 2013; Maes et al. 2014).
Central in these approaches is the idea of circularity between stimulus and reaction, in the sense that a situation as perceived leads to an activity that is evaluated in terms of its beneficial or expected results. What matters in this view are not the actions proper but their results. Playing a musical instrument, e.g., is a typical example of motor output, which becomes a behavioral response to perceptual input, as soon as there is a modification or adjustment of the sound production as a result of feedback through the senses. What counts is the possibility of comparing actual sounds with a target performance which is present already in imagery.
Sense-Making and the Role of Subjectivity : Affordance and Functional Tone
Von Uexküll’s writings have been influential in the fields of biosemiotics and theoretical biology. His idea of circularity, e.g., has a lot of descriptive and explanatory power by emphasizing the role of functional significance in knowledge construction. It has furthered to some extent also the boost of action and perception studies. By stressing the role of knowledge construction, he anticipated also on the major claims of second order cybernetics, which typically conceives of the observer as a participant and as part of the observed system. Such an approach revalues the role of subjectivity in sense-making with a shift from mere communication and control to the role of interaction (Brier 1999). Translated to the realm of music, this means that a listener has a lot of freedom in the delimitation of the elements that he/she considers worthy of attention. It is possible, however, to reduce the virtual infinity of possible elements to manageable proportions by relying on ecological and biological constraints on coping with the sounds. The former are related to the sensory range of the senses with only a limited set of distinctions and observables being accessible for processing. The latter are related to the mechanism of event perception and the concepts of affordance and functional tone.
The first – the concept of functional tone – was introduced by von Uexküll to describe the transformation of neutral objects into objects that function as meaning-carriers as soon as they enter into a relationship with a subject. This is demonstrated clearly in his example of an angry dog that is barking at a person on a country road. In order to drive away the dog, the person picks up a stone and throws it at the dog. The stone, in that case, changes its meaning from a “path-quality” to a “throw-quality”. It first lay on the ground, incorporated in the road to serve as a support for the walker’s feet. As soon, however, as it was picked up to throw it at the dog, it became a missile and a new meaning was imprinted on it (von Uexküll 1982 [1940] : 27). Another clarifying example is that of a tree that has a number of different qualities or tones, dependent on the intentions that an animal or human being confers on it. It can be a shelter for a fox, a support for the owl, a thoroughfare for the squirrel or a source of valuable raw material for the forester (von Uexküll 1957 [1934]). It clearly demonstrates that there is no one-to-one relationship between an object in the outer world and its actual meaning. The way an organism perceives the world, in fact, is determined by a network of functional relations, which together constitute its phenomenal world or Umwelt. It means that the number of objects an organism can distinguish is equivalent to the number of functions it can carry out. Its subjective world, in that view, is the outcome of experiences with each new experience entailing a readjustment to new impressions so that new functional tones can be created (von Uexküll, 1957 [1934] : 49).
What matters in this approach, is the sensitivity to functional characteristics of the environment. It means that animals or organisms in general perceive objects in the environment in terms of what they afford for the consummation of behavior rather than in terms of their objective perceptual features. It is a conception that is related to Gibson’s concept of affordance (Gibson 1966; see also Chimera 2003). Affordances, in his view, are environmental supports for the intentional activities of an organism, which can be defined as the perceived functional significance of an object, event or place. They are subjective qualities that render them apt for specific activities such as supporting locomotion, concealment, manipulation, nutrition and social interaction (Gibson 1979), but they are not merely subjective qualities. Relying on objective environmental features of the world as well as on perceiver-specific qualities, they go beyond the subjective/objective dichotomy (Heft 2001).
Applying this to the realm of music should mean that sounding events must be understood in terms of what they “afford” to the listeners, rather than in terms of their acoustic qualities. The question, however, is to define what these musical affordances are. A rather obvious answer is related to the possibility to generate sound on the basis of raw material that is to be found in the external and internal environment. As such, two major possibilities can be considered : the production of musical instruments out of sounding material and the development of techniques for sound production and modulation of the sound (Reybrouck 2012). As to the first, there is a whole history of instrument building, which was one prolonged search for applying craftsmanship to raw materials in order to produce musical sounds. The development of techniques for playing, on the other hand, embraces a while gamut of sound producing actions such as hitting, stroking, kicking and blowing, which can be refined further to modulate the sound. Strings, e.g., can be plucked or bowed, and even within the action category of bowing, there is a whole spectrum of shaping of the sound.
Affordances, further, are not restricted to the activity signature of musical sounds. It is possible, in fact, to adopt a view that conceives of musical affordances also at the receptive level of experience, embracing perceptual qualities, mood induction qualities and socio-communicative qualities, which invoke aspects of sense-making, emotional experience, aesthetic experience, entrainment and judgments of value (Krueger 2011; Windsor 2004).
Musical affordances can thus be conceived either at the productive and the receptive level. The former did receive most attention up to now. It is possible, however, to bring together both approaches as exemplified in the huge body of action and perception studies (Hummel et al. 2001; Leafed et al. 2017; Print & Chatter 2005). There is a distinction, however, between the action aspects of sound production and what this affords to the listener. To conceive of music in terms of experience involves, in fact, an aspect of egocentricity that describes subjective experiences in terms of bodily resonance or motor imagery that projects our bodily movements to the music. Such resonance can be considered as a phenomenal experience which involves the experience of movement but without the action being manifest. It corresponds to the so called “internal imagery” or “first person perspective” which enables the transition from overt action to internalized forms of action. The whole process calls forth a kind of motor empathy, allowing listeners to experience the music as something that moves over time, while simultaneously experiencing this movement as a movement of the own body (Reybrouck 2001b). As such, motor components seem to be involved in perception and are an integral part of it (Mahon 2008; Noë 2004). Music, in this view, can be perceived in terms of its motor induction capacities, which means that music involves entrainment and the possibility to move in reaction to the sounds either in ways that are motivated in their relations to the sounds or in a more general way as forces and energies which account for the perception and imagination of tension, resolution and movement. These movements, finally, can be overt and manifest but they can operate at virtual levels of imagery and simulation as well. As such, one could arguably extend the action-related affordances to four major categories : the sound producing actions proper, the effects of these actions, bodily resonance and movements that may be induced by the sounds (Reybrouck 2012, 2017a).
Conclusion and Perspectives
The main aim of this paper was to provide an operational description of the way how listeners deal with music-as-heard. Starting from a definition of music as sonic environment, it argued for a broadening of scope from a restricted body of musical works to an approach that encompasses all possible kinds of sounds. Every listener, in this broader view, builds up relations with the sonic world, selecting some of the sounds to give them special meaning. As such, he/she constructs a sonic Umwelt, as the sum total of subjective meanings that are assigned to those elements that receive semantic weight. Musical sense-making, then, relies on music knowledge construction that must be generated as a tool for adaptation to the sonic world. It is a viewpoint that can been described in ecological terms as coping with the sounds if we conceive of listeners as organisms and of music as environment.
The ecological approach is challenging. It has the advantage of broadening the scope of sense-making by embracing all kinds of sonorous events into its theoretical framework. By stressing the role of functional significance and interactions with the world, it is also on a continuum with the biosemiotics claim of construction of an inner model of the world. Musical sense-making, in this view, holds a position between bottom-up and top-down processing. Starting from biological and dispositional constraints, which are dependent on the possibilities and limitations of the sensory system and the processing power of the brain, it can rely on several levels of processing, going from low-level reactivity to higher-level processing by the brain. Both levels are not opposed to each other but are complementary to some event with a dynamic tension between discrete and continuous processing of the sounds. The level of sensory processing, in fact, is continuous, but sense-making relies on principles of cognitive economy, substituting discrete events with unit character for temporal extensions that are continuous in their unfolding. It thus seems that musical sense-making holds a hybrid position in between several ill-defined dichotomies such as the bottom-up/top-down approach, the discrete/continuous processing and the subjective/objective dichotomy. Some of these issues are the subject of ongoing research, but much more efforts are still needed to resolve this in a satisfactory way (Reybrouck 2017b).
Parties annexes
Biographical note
Mark Reybrouck graduated in physical education, physical therapy and musicology. He is currently professor of music at the University of Leuven (Belgium) and visiting professor at Ghent University (Belgium). His major research interests are music semiotics and listening strategies, both at a theoretical and empirical level of research. His theoretical contributions concern: musical epistemology, with a major focus on musical sense-making in real-time listening situations; the related topic of music as experience; and the embodied and enactive approach to music cognition. As such he has explored the biological foundations of musical epistemology and the possibilities of biosemiotics and ecological perception and has made major contributions to translate them to the realm of music. Besides this theoretical work he has been involved also in empirical research on representational and metarepresentational competence of children in music listening tasks. His most recent research agenda is related to musical emotions and music and brain studies.
Bibliography
- Balkwill, L.-L. (1997) Perception of Emotion in Music : A Cross-Cultural Investigation. Unpublished doctoral dissertation. North York : York University.
- Balzano, G. (1986) “Music Perception as Detection Of Pitch-Time Constraints”. In V. McCabe & G.Balzano (Eds.), Event Cognition : An Ecological Perspective. Hillsdale, NJ – London : Lawrence Erlbaum : 217-233.
- Barrett, L. (2011) Beyond the Brain. How the Body and Environment Shape Animal and Human Minds. Princeton – Oxford : Princeton University Press.
- Bartlett, J. (1984) “Cognition of Complex Events : Visual Scenes and Music”. In W. Crozier & A. Chapman (Eds.), Cognitive Processes in the Perception of Art Amsterdam - New York - Oxford : North-Holland : 225-251.
- Bootz, P., & Hautbois, X. (2007) “Time Measures in Documents : The Model of ‘motifs temporels paramétrés’” In R. Skare, N. W. Lund, A. Vårheim (Eds.), A Document (Re)turn. Frankfurt am Main : Peter Lang : 197-222.
- Brier, S. (1999) “Biosemiotics and the Foundations of Cybersemiotics”. In Semiotica 127(1/4) : 169-198.
- Bruner, J., Goodnow, J., & Austin, G. (1956) A Study of Thinking. New York : Wiley.
- Burianová, H., Marstaller, L, Sowman, P., Tesan, G., Rich, A., Williams, M., Savage, G., & Johnson, B. (2013) “Multimodal Functional Imaging of Motor Imagery Using a Novel Paradigm”. In NeuroImage (71) : 50-58.
- Cassidy, A., & Einbond, A. (2013) Noise in and as Music. Huddersfield : University of Huddersfield Press.
- Chemero, A. (2003) “An Outline of a Theory of Affordances”. In Ecological Psychology (15) : 181–195.
- Cogan, R. (1984). New Images of Musical Sound. Cambridge (MA), London : Harvard University Press.
- Colombetti, G. (2007) “Enactive Appraisal”. In Phenomenology and the Cognitive Sciences (6) : 527-546.
- Cumming, N. (2000) The Sonic Self : Musical Subjectivity and Signification. Bloomington : Indiana Universtiy Press.
- Dubois, D. (1991) Sémantique et cognition. Catégories, prototypes, typicalité. Paris : Editions du CNRS.
- Eidsheim, N.S. (2015) Sensing Sound. Singing & Listening as Vibrational Practice. Durham – London : Duke University Press.
- Frey, A., Hautbois, X., Bootz, P., & Tijus, C. (2014) “An Experimental Validation of Temporal Semiotic Units and Paramtereized Time Motifs”. In Musicae Scientiae (18)1 : 98-123.
- Gaver, W.W. (1993) “What in the World Do We Hear? An Ecological Approach to Auditory Event Perception”. In Ecological Psychology (5)1 : 1-29.
- Gibson, J. (1966) The Senses Considered as Perceptual Systems. Allen & Unwin, London.
- Gibson, J. (1979) The Ecological Approach to Visual Perception. Boston : Houghton Mifflin Company.
- Haeckel, E. (1988[1866]) Generelle Morphologie des Organismus, Bd. 2 : Allgemeine Entwicklungsgeschichte. Berlin : de Gruyter.
- Handel, S. (1989) Listening. An Introduction to the Perception of Auditory Events. Cambridge - London : MIT Press.
- Heft, H. (2001) Ecological Psychology in Context : James Gibson, Roger Barker, and the Legacy of William James’s Radical Empiricism. Mahwah, NJ : Lawrence Erlbaum Associates.
- Hommel, B., Müsseler, J., Aschersleben, G., & Prinz, W. (2001). “The Theory of Event Coding (TEC) : A Framework for Perception and Action Planning”. In Behavior and Brain Sciences (24)6 : 849–878.
- Hurley, S. (2002) Consciousness in Action. Cambridge (MA) – London : Harvard University Press.
- Hutto, D., & E. Myin (2013) Radicalizing Enactivism. Basic Minds without Content. Cambridge (MA) : The MIT Press.
- James, W. (1950 [1890]) The Principles of Psychology. Vol.2. New York : Dover.
- Krueger, J. (2011) “Doing Things with Music”. Phenomenology and the Cognitive Sciences (10) : 1-22.
- Kull, K. (1998) “On Semiosis, Umwelt, and Semiosphere”. In Semiotica, (120)3/4 : 299-310.
- Lakoff G. (1987) Women, Fire, and Dangerous Things : What Categories Reveal About the Mind. Chicago : University of Chicago Press.
- Lesaffre M., Maes, P.-J., & Leman M. (Eds.) (2017) The Routledge Companion to Embodied Music Interaction. New York : Routledge.
- Lombardo, T.J. (1987) The Reciprocity of Perceiver and Environment. The Evolution of James J. Gibson’s Ecological Psychology. Hillsdale, NJ - London : Lawrence Erlbaum.
- Maes, P.-J., Leman, M., Palmer, C., & Wanderley, M. (2014) “Action-Based Effects on Music Perception”. In Frontiers in Psychology (4) : 1008.
- Mahon, B. (2008) “Action Recognition : Is It a Motor Process?” In Current Biology (18)22 : R1068-R1069.
- Maiese, M. (2011) Embodiment, Emotion, and Cognition. New York : Palgrave Macmillan.
- Maschke, C., Rupp, T., & Hecht, T. (2000) “The Influence of Stressors on Biochemical Reactions – A Review of Present Scientific Findings with Noise”. In International Journal of Hygiene and Environmental Health (203) : 45-53.
- McAdams, S. (1984) “The Auditory Image : A Metaphor for Musical and Psychological Research on Auditory Organization”. In W. Crozier, W. R. & A. J. Chapman (Eds.), Cognitive Processes in the Cognition of Art. North-Holland : Elsever Science Publishers : 289-323.
- Menary, R. (2010)“Introduction to the Special Issue on 4E Cognition”. In Phenomenology and the Cognitive Sciences (9) : 459-463.
- Meystel, A. (1998) “Multiresolutional Umwelt : Towards a Semiotics of Neurocontrol”. In Semiotica (120)3/4 : 343-380.
- Michaels, C., & C. Carello (1981) Direct Perception. Englewood Cliffs, New Jersey : Prentice-Hall.
- Neisser, U. (Ed.) (1987) Concepts and Conceptual Development : Ecological and Intellectual Factors in Categorization. Cambridge : Cambridge University Press.
- Noë, A. (2004) Action in Perception. Cambridge : MIT Press.
- Prinz, W., & Chater, N. (2005) “An Ideomotor Approach to Imitation”. In S. Hurley (Ed.). Perspectives on Imitation : From Neuroscience to Social Science. Vol.1 : Mechanisms of Imitation and Imitation in Animals. Cambridge, MA : MIT Press : 141-156.
- Reybrouck, M. (2001a) “Biological Roots of Musical Epistemology : Functional Cycles, Umwelt, and Enactive Listening”. In Semiotica (134)1/4 : 599-633.
- Reybrouck, M. (2001b) “Musical Imagery Between Sensory Processing and Ideomotor Simulation”. In R. I. Godøy & H.Jörgensen (Eds.), Musical Imagery. Lisse : Swets & Zeitlinger : 117-136.
- Reybrouck, M. (2005) “A Biosemiotic and Ecological Approach to Music Cognition : Event Perception Between Auditory Listening and Cognitive Economy”. In Axiomathes. An International Journal in Ontology and Cognitive Systems (15)2 : 229-266.
- Reybrouck, M. (2012) “Musical Sense-Making and the Concept of Affordance : An Ecosemiotic and Experiential Approach”. In Biosemiotics (5)3 : 391-409.
- Reybrouck, M. (2013) “Musical Universals and the Axiom of Psychobiological Equivalence”. In J.-L. Leroy (Ed.), Topicality of Musical Universals / Actualité des Universaux musicaux. Paris : Editions des Archives Contemporaines : 31-44.
- Reybrouck, M. (2015) “Music as Environment : An Ecological and Biosemiotic Approach”. In Behavioral Sciences (5)1: 1-26.
- Reybrouck, M. (2017a) “Music Knowledge Construction. Enactive, Ecological, and Biosemiotic Claims”. In Lesaffre M., Maes P.J., Leman M. (Eds.), The Routledge Companion to Embodied Music Interaction. New York : Routlege : 58-65.
- Reybrouck, M. (2017b) Perceptual Immediacy in Music Listening : Multimodality and the “In Time/Outside of Time” Dichotomy. Versus (to appear)
- Reybrouck, M., & Brattico, E. (2015) “Neuroplasticity Beyond Sounds : Neural Adaptations Following Long-Term Musical Aesthetic Experiences”. In Brain Sciences (5)1 : 69-91.
- Reybrouck, M., & Eerola, T. (2017) “Music and Its Inductive Power : A PsychoBiological and Evolutionary Approach to Musical Emotions”. In Frontiers in Psychology (8) : 494.
- Rosch, E., & Lloyd , B. (Eds.) (1978) Cognition and Categorization. Hillsdale, New York : Erlbaum.
- Schiavio, A., van der Schyff, D., Cespedes-Guevara, J., & Reybrouck, M. (2016) Enacting Musical Emotions. Sense-Making, Dynamic Systems, and the Embodied Mind. Phenomenology and the Cognitive Sciences. Published online : 22 July 2016.
- Shaw, R., & W. Hazelett, (1986) “Schemas in Cognition”. In V. McCabe & G. Balzano (Eds.). Event Cognition : An Ecological Perspective. Hillsdale, NJ - London : Lawrence Erlbaum : 45-58.
- Shaw, R., & Pittenger, J. (1978) “Perceiving Change”. In H. L. Pick & E. Salzman (Eds.), Modes of Perceiving and Processing Information. Hillsdale, New Jersey : Erlbaum : 187-204.
- Shaw, R., O. Flascher, & W. Mace (1996) “Dimensions of Event Perceptio”. In W. Prinz & B. Bridgeman (Eds.), Handbook of Perception and Action. Vol. 1 : Perception. London, San Diego, New York, Boston, Sydney, Tokyo, Toronto : Academic Press, Harcourt Brace : 345-395.
- Sloboda, J. (2005) Exploring the Musical Mind. Cognition, Emotion, Ability, Function. Oxford : Oxford University Press.
- Standley, J. (2002) “A Meta-Analysis of the Efficacy of Music Therapy for Premature Infants”. In Journal of Pediatric Nursing (17)2 : 107-113.
- Stern, W. (1897) “Psychische Präsenzzeit”. Zeitschrift für Psychologie, XIII : 325.
- Stevens, S. (1951) “Mathematics, Measurement, and Psychophysics”. In S. Stevens (Ed.), Handbook of Experimental Psychology. New York-London : Wiley - Chapman & Hall : 1-49.
- Stewart, J., Gapenne, O., & Di Paolo, A. (2010) Enaction. Toward a New Paradigm for Cognitive Science. Cambridge (MA) : MIT Press.
- Tagg, P. (2013) Music’s Meanings. A Modern Musicology for Non-Musos. New York & Huddersfield : The Mass Media Music Scholars’s Press.
- Tenney, J., & Polansky, L. (1980) “Temporal Gestalt Perception in Music”. In Journal of Music Theory (24) : 205–21.
- Thompson, E. (2005) “Sensorimotor Subjectivity and the Enactive Approach to Experience”. In Phenomenology and the Cognitive Sciences (4) : 407–427.
- Thompson, E. (2007) Mind in Life : Biology. Phenomenology, and the Sciences of Mind. Cambridge : Harvard University Press.
- Uttal, W. (1973) The Psychobiology of Sensory Coding. New York - Evanston - San Francisco - London : Harper & Row.
- Uttal, W. (1978) The Psychobiology of Mind. Hillsdale, NJ : Lawrence Erlbaum.
- Varela, F., Thompson, E., & Rosch, E. (1991). The Embodied Mind. Cognitive Science and Human Experience. Cambridge (MA), London : MIT Press.
- Voegeli, S. (2010) Listening to Noise and Silence : Towards a Philosophy of Sound Art. London : Bloomsbury Publishing.
- von Uexküll, J. (1957 [1934]) “A Stroll Through the Worlds of Animals and Men. A Picture Book of Invisible Worlds”. In C. Schiller (Ed.), Instinctive Behavior. The Development of a Modern Concept. New York : International Universities Press : 5-80.
- von Uexküll, J. (1982 [1940]) “The Theory of Meaning”. In Semiotica (42) : 25-82.
- Wagner, M (1994) Introductory Musical Acoustics. Raleigh, NC : Contemporary Publishing.
- Windsor, W.L. (2004) “An Ecological Approach to Semiotics”. In Journal of the Theory of Social Behaviour (34)2 : 179-198.
- Wittmann, M., & Pöppel , E. (1999-2000) Temporal Mechanisms of the Brain as Fundamentals of Communication - with Special Reference to Music Perception and Performance. Musicae Scientiae, Special Issue : 13-28.
- Wittmann, M. (2011) “Moments in Time”. In Frontiers in Integrative Neuroscience (5) : 66.
- Ziemke, T., & Sharkey, N. (2001) “A Stroll Through the World of Robots and Animals : Applying Jakob von Uexküll’s Theory of Meaning to Adaptive Robots and Artificial Life”. Semiotica (134)1/4 : 701-746.