Corps de l’article

Introduction

Humans have settled at all ‘corners’ of the globe and have successfully adapted to a range of different environments, using a myriad of extrasomatic means of adaptation (Binford 1962). A relatively recent technological adaptation is the development of sensing instruments to sense the world beyond human somatic/sensorial limits. Boundaries of distance and sensing range are thereby expanded into so-called ‘unknown territory’. This has opened up new ways of exploring ontological and physical limitations, yet these and other technological developments of the last hundred years or so have also created an unprecedent potential for interfering with communication processes beyond the human sphere.

Understanding different ways of experiencing the world requires the exploration of the limitations of the human mind through, for instance, the – multimodal – languages we use to make sense of, represent, and navigate our surroundings. Languages differ among and within species; humans speak different languages and use different modes of expression. Other species, notably bees, are known to communicate with a (place) specific ‘dialect’ to successfully navigate their world (von Frish 1970). Yet the common assumption is that humans are unique in their ability to manipulate symbols, which is often considered a sign of the superiority of the human mind (Deacon 2010). However, I contend that the connection between representation and perception of the environment is in need of reassessment. Indeed, I believe that the idea that symbolic communication infers the understanding of underlying indexical relationships, as discussed by Deacon (2010), is questionable from the perspective of spatial knowledge. Furthermore, as argued by Pickering (2005), the capacity to (symbolically) represent the world, crucial to what it is to have a human mind, has been mistaken for what it means to have any sort of mind at all. Besides, symbol use in animals is not unknown. For instance, Von Frish (1970) conducted numerous field experiments demonstrating that individuals of a bee family can perform a symbolic dance to direct hive members toward food sources with great accuracy and precision. Once we get a glimpse of this phenomenon and become aware of this incredible feat, the question arises as to how many other amazing acts of communication are performed by our fellow organisms, even within or right outside the human sensory range, scale, and/or awareness.

Since the basic unit of biotic being is the organism-in-its-environment (Ireland 2015), the lack of long-term studies conducted in ecologically relevant settings has been identified as a reason for the current gap in our scientific knowledge regarding such phenomena and relationships (Spillivalli et al. 2011).

Spatial Configurations in Humans and Bees

Researchers who currently work within the parameters of biosemiotics come from different disciplinary backgrounds yet share an interest in the myriad communications between living organisms (Favareau 2010). Humans, traditionally studied within the field of anthropology, are but one component in a larger semiotic network, and this network can be thought of as overlapping subjective perceptual worlds that together constitute the intersubjective, knowable world (Birnbaum 2008). Yet, questions arise, such as : 1) How are these worlds spatially configured? Or, what is the spatial structure of sensory stimuli and information in each organism’s environment? And 2) What is the (multimodal) constellation of the (perceptual) modalities used by an organism to communicate in its world and how do these sensory configurations interact within and among shared worlds?

What I mean by spatial configuration is the spatial relationship between source, path, and receiver of a specific modal object. For instance, a sound event that originates at a specific location, (e.g. a bird call) travels along a path through a medium that in turn impacts the intensity and distance at which the signal can be received by sensing organisms and instruments. Communication between sound emitter and receiver/organisms, and the meaning attributed to the signal in this relationship is the focus of biosemiotic investigation. Studies of bird songs specifically have shown that birds use particular frequency bands to communicate their message (Krause & Farina 2016). The spatial configuration of this and other multimodal communication networks are an important, but often overlooked aspect for understanding the dynamics and ecology of the communication sphere across modalities.

My interest and research efforts focus on (the history of) human participation in this network and the ability to gain ambient knowledge from a position in the land through multimodal signals in our surroundings (ground-truthing). Driven by a widely-held assumption that the visual sense is the most important in human perception, the initial focus of my research was on visual-spatial thinking. However, I have come to revisit this position due to recent work that demonstrates diversity in spatial knowledge among human groups, including differences in spatial ontology and frames of reference used by different groups. These new insights underscore the importance of engagement with, and experience within the environment. In other words, groups that live in non-urban/rural settings use different categories of space than related groups in urban settings. This is suggested to be strongly influenced by the need to navigate that environment, such as categories of wind, objects that express the relationship between topography and atmospheric circulation. Such categories of space are absent in urban-based ontologies (Palmer 2015; van der Elst 2010).

Now briefly consider the Western honey bees (Apis mellifera), buzzing around scented flowers to forage food using their dead-reckoning spatial skills. We easily forget that these are not just individuals; collectively they make up a super organism, in which queen, worker and drone each perform a specific function. Human survival is highly dependent upon this organism to do its work of plant pollination. The bee population is symbiotically related to the aromatic plant world. The flower is dependent on the bee to perform pollination and sends out an aromatic and color signal to attract it. In return the flower provides the bee with food/nectar, which is then transformed by the bees into honey, wax and propolis. Humans benefit from this relationship in more ways than one; pollination is considered an ecosystem service (Millenium Ecosystem Assessment 2005) that ensures a significant amount of the world’s food supply, given that flowering plants can produce fruit suitable for human consumption. Furthermore, honey has been an important food source for humans from time immemorial. The earliest depiction of honey harvesting is from a rock art painting in Spain, dating back 8,000 years, but an interesting hypothesis is put forth that suggest pushing this date back to 40,000 years. Thus, Crittenden proposes that may have had a significant role in developing large brains, related to first tool making (2011). In any case, honey production forms an incredible cycle and network where humans play a recipient part. Contemplating the importance of this relationship provides us with a different perspective on our surrounding world, a relational semiotic network, or better yet a semiosphere (Lotman & Clark 2005).

Although research is still limited, it is estimated that the increasing air pollution since the Industrial Revolution has significantly diminished the travel distance/reach of aromatic flower signals (McFrederick et al. 2009). This has dramatically impoverished the bee’s semiosic world, diminishing meaningful objects that help it (and by association humans) survive.

At the same time as pollution is reaching critical levels, as measured for humans, research is mounting that shows that many animals possess sensory abilities that humans lack as an organism (e.g. Chittka & Brockmann 2005). This points to a collective-intersubjective world that reaches beyond human capability or even imagination. Furthermore, it implies that many objects in our surroundings may be recognized by, and be meaningful to other organisms, but not to humans, objects that are impacted by changes in atmo- and hydrosphere. For instance, UV light presents a clear navigable path for bees in the atmosphere toward their food source, yet it is undetectable to the human sensorium (von Frish 1970).

Even though humans are all equipped with the same sense organs, the ratio of use of the sensory modalities within a specific cultural constellation differs across the human population (Kress 2010). Kress further argues that (natural) language cannot cover all meaning and that the different modes of communication vary in importance in different cultures. This can result, among other things, in different ontologies of space that underpin a specific model of the world (Levinson 2003). For instance, a strong focus on oral language use can indicate an emphasis on auditory aspects, whereas focus on written language likely emphasizes the visual domain. The dominance of the visual sense in our current scientific knowledge system has arguably led to an underappreciation of the role of other sensory sources of information and modes of communication in our shared environment (e.g. Krause 1983; Horowitz 2013; Schafer 1994). Briefly put, within sensory modalities shared by many animals, such as vision, hearing, and olfaction, there are significant differences between species in range, resolution, and meaning covered. Moreover, being equipped with the same kinds of sensing organs and ranges does not mean that the sum of perceptual fields results in identical subjective worlds, for the subjective world is dependent on a number of factors, including the kind of environment in which an organism lives (Palmer 2015).

The idea that the human mind differs from all other sentient beings is problematic; yet a specific notion to separate the human mind from our bodies has guided modern scientific inquiry since its inception and created a research impasse. This is based on the Cartesian notion that although the exterior world is grasped through the mechanical work of the senses, an immediate entity, a concept or idea is necessary to stand between the outside world (reality) and the mind. In this way, the human mind no longer has direct access to the world. Whereas other beings are supposed to connect through the mechanics of their sensory organs, humans are thus supposed to ‘translate’ sensory stimuli into a conceptual world that underpins thoughts and action. A common objective in biosemiotic studies is thus to transcend this impasse; a central tenet of research within the parameters of biosemiotics considers all organisms as connected to their world in a relational semiotic network and conceives of the mind as embodied (Hoffmeyer 2012).

Even though it was Thomas Sebeok who was instrumental in establishing biosemiotics as a field of research, it is well known that its fundamental principles were laid out much earlier in the 20th century in the writings of von Uexküll (biology) and Peirce (semiotics) (Favareau 2010). Resting on Uexküll and Peirce, we can establish the link between biosemiotics and spatial information theory/spatial science in order to investigate the boundaries of our spatial thinking that underpins our engagement with the world. Indeed, Peirce proposed that all forms of thought (ideas) are essentially communication (transmission of signs), organized by an underlying logic (semiotic) that is not fundamentally different for communication processes inside or outside the brain. In this way he rephrased the problem of mind in terms of communication (Hoffmeyer 2012). Basically, things don’t mean anything unless interpreted. Also focused on communication but from a biological perspective, von Uexküll (2010) introduced the term/concept of Umwelt to describe the subjective universe of species underpinning his theory of meaning. In other words, the world that animals (including humans) perceive is not an objective world, but a product of the particular sense organs that each species acquired in its evolutionary history. Uexküll’s main concern was the shortcoming of the “biological machine” approach to the investigation of living systems. The concept of Umwelt thus articulates, as stated by Favareau, a “set of agent-object relations reducible neither to the organization of the subject nor to the organization of the environment but always as the product of the interaction between the two. […]and these relations of seemingly ‘private and subjective experiences’ are examinable […] by science” (Favareau 2010 : 83). The conceptions put forth by Peirce and von Uexküll were, and still are quite revolutionary, as they closely resemble foundational ideas of a number of current innovative research directions in spatial thinking.

Ultimately, perceptual and cognitive processes have developed through evolution to enable individuals to act in the environment and achieve a series of goals to survive (Clifton et al. 2016). According to Pickering, embodied action, not language or mental representation, is both the ontogenetic and phylogenetic origin of mental life (Pickering, 2005 : 198). The mind body dichotomy that divorced many of us from direct access to the surrounding world, presents us tangentially with the following related presumptions that are questioned in this paper : 1) humans are superior beings because of their ability to use symbols and, 2) spatial reasoning and knowledge is based on visual perception and therefore we do not commonly consider objects in other sensory domains as essential to spatial reasoning and navigation.

By linking insights from biosemiotics, spatial sciences and anthropology we should be able to address the problem of the disconnectedness between the human mind and the – multisensory – physical world (Hoffmeyer 2012; Pickering 2005; van der Elst 2010) through exploration of boundaries of – elements of – experiential spaces. Let us specifically focus on new insights in spatial sciences as a means to investigate the human-land relationship.

Geocentric Frame of Reference

Spatial cognition is considered a fundamental cognitive domain, it forms the foundation for the way we take part in, navigate, and design our worlds (Clifton et al. 2016; Levinson 2003). This is considered true for humans, yet spatial intelligence is not something that is often attributed to other species, as modern science is rooted in the machine metaphor in which an organism’s behavior in the natural world is primarily understood as a stimulus and response system, instead of originating from intention. The theory of meaning put forth and published by von Uexküll in 1940 presents an early effort to address the shortcoming of conventional science in dealing with meaning and intentionality in biology, but it is only a recent development that researchers are beginning to take this idea more seriously. Studies that consider plant communication and plant intelligence are on the rise, suggesting that a re-assessment of anthropocentrism is possible, and spatial intelligence is occurring across living organisms (Holopainen en Blande 2012).

As an overarching field, spatial information science not only addresses what landscape/spatial objects are (ontologically speaking), but also how we represent these objects and analyze their relationship in space and time. Over the last two decades, the research focus has broadened from representing space geometrically to encompass investigations into spatial reasoning, spatial language and human experience of space (Egenhofer & Golledge ed. 1998; Tversky 2003). Notably, research in psycholinguistics has demonstrated the diversity in spatial thinking among different language groups and thereby has opened up new avenues to investigate spatial experience (Levinson 2003; Mark et al. eds. 2011). Yet modeling spatial perceptual structures, that is to say environments in which the relationships between sign, object, interpretant and medium/facilitator can be sketched out, is still in its infancy. A term such as ‘windgap’ is impossible to map using current spatial information systems. Although the term ‘windgap’ can be considered as a reference to a topographic feature (e.g. a valley), the important relationship referred to by this term is the windgap’s ability to mediate air flow, which can then be interpreted in specific meaningful way by an organism (Harrington 1916; van der Elst 2010).

The pervasive notion that humans employ two basic Frames of Reference (FoR) for spatial knowledge acquisition (a relative/ego-centric FoR and an intrinsic/object-centered FoR), underpins the idea of universality in human spatial cognition. Recent research however upended the idea that all humans experience space and gain spatial knowledge in the same way, and as it turns out, a third, absolute/geocentric FoR is used in a number of language groups, dispersed across different geographical regions and associated with different spatial ontologies (Majid et al. 2004). The different categorizations of space can also reveal diverging ideas concerning how objects and events are defined and related. These findings imply that there are no innate universal spatial categories in human cognition. However, basing himself on the comparison of different languages that are spoken in diverse topographic environments and settings (i.e. rural, urban), Palmer (2015) argues that cross-cultural evidence points to innate human responses to the environment. “Wind”, for instance, is an important component of landscape in several cultures, and categories often blur the boundaries of symbol and index. For instance, for some communities ancestor spirits are carried by the wind; spirit also means breath, and as such it indicates a (physical) life-giving force. Moreover, winds across Artic Finland strongly influence the direction taken by reindeer herds who use it in smelling food sources (Leena Valkeapää, personal communication). In general, within a geocentered frame of reference, the speaker’s view is irrelevant to the encoding and decoding of spatial relations, but variation exists in how space is used.

These preliminary findings support the idea of a prominent role for embodiment in developing spatial skills, and the importance of connectedness in the physical world in forming spatial knowledge (Clifton et al. 2016; Hegarty 2006; Palmer 2015). In this world the human is a component, but not its focal point. This encourages further exploration into the possibility of an ecologically based approach to perception (Gibson 1979) employing a geo-centered frame of reference, which presents a new direction in humanistic geospatial approaches.

To illustrate, again, imagine briefly the parts of the semiotic network in which humans are connected with bees and flowering plants. In this network it is difficult to determine if there is a central subject or object that is more important or salient than any other. The navigable path in the atmosphere that leads the worker bee to her food source, may be a suitable axis in a coordinate system to understand the relationship; yet it makes little sense to the (modern) human. We are unable to detect the relevant signal (UV – polarized) using our sensory organs. However, considering the importance of these signals can encourage the development of sustainable technologies that detect them and serve as indices for the underlying framework of source, medium and message pertaining to them.

Even though focus has shifted in the development of spatial technologies to encompass a humanistic approach and ways of modeling the human experience, these efforts have primarily considered visual perception as the most important sense underpinning spatial thinking. Based on Gibson’s ecological approach to perception, Higuchi developed a systems approach to (human) visual perception of the landscape during the late 1970s, for which he defined nine indices (Gibson 1979; Higuchi 1983). It is currently integrated within Geographic Information Systems (GIS) as viewshed analysis. Indices include, among others : line of sight, distance zones, angle of elevation and light. These indices point to, or approximate the spatial structure of visual perception. Briefly, visual perception is based on the relationship between source of light (sun) and sensor (eye), whereby the light interacts with material surfaces that reflect, refract, or absorb the light. This reflected light could be received by the sensor organ (eye) or instrument, such as those used in obtaining satellite imagery. Using Higuchi’s indices, the visual field from the sensor position can then be calculated, but it is different from the perceived field. To my knowledge, Higuchi’s pioneering work has not yet led to the development of comparable multisensory-based systems at a landscape scale (Basdogan & Loftin 2008; van der Elst et al.). Indices of spatial structures of perception remain a challenge.

Building on Higuchi’s work, in my previous studies with Tewa placenames (Harrington 1916) I have used the concept of “cognitive landscapes”, building on principles that pertain to remote sensing and geosciences, to explore human land relationships. In short, this work was based on the concept of visual sense and on perception as remotely sensed information and how, together, they can be used to “code” an image layer based on Tewa placenames within a geographic information system (GIS). The system also contains standard landscape data, and thus enables the integration of perceptual (coded placenames) and environmental data (van der Elst 2010). One of the goals of this research was to test the potential of current spatial representation systems to map spatial categories that originate within a different knowledge system and understand their meaning within their original cultural landscape context. The use of remotely sensed data provides an interesting reference, since it is collected at time intervals and therefore changes or fluidity (of aspects) of the landscape can be considered. For instance, each plant species shows a unique spectral signature over time, a seasonal pattern. As a result, a more nuanced comparison between land and thinking about land is made possible, appreciating the more ephemeral qualities (and quantities) that make up our surroundings. Connection to the land is then about place, a continuous changing of relationships (between organisms in their environment).

Place names are of interest because they take an ontological position between proper names and landscape categories (Cablitz 2008), thus providing insight into the nature of the human relationship to the physical world. As it turns out, while many names/terms are related to the environment, not all names/categories are visually based, and/or are based on objects more transient in character. Different sounds of wind for instance, are located within a sphere (atmosphere) that is not included within current spatial information systems. In his earlier work, Krause describes a conversation in the land in which his interlocutor shows him how the wind ‘plays’ the vegetation. This was recognized by Krause (1993) as an example of the origin of music, yet with a clear place-based message in the land.

Plants as instruments that are not modeled, but topographical features that are commonly represented with spatial systems (Earth surfaces) can thus be thought of as mediators of sound objects or events. However, these representations are only parts of an object or event, which means that we overlook the plants, the wind, and the atmosphere in general. This brings us to another salient feature of conventional mapping, in which above surface areas are considered ‘empty space.’ My research brought to the fore that in order to understand perceptual/semiotic space it is necessary to consider different spatial perceptual structures and ontologies of space that include this space as part of the semiotic space. The main outcome is that that objects can no longer be considered to exist exclusively in the visual domain in human experience and spatial reasoning. The spatial structure that underpins current spatial – visually dominated – representation systems is therefore insufficient to represent the totality of (human) spatial experience. Field visits of locations made it clear for me that an embodied approach should be a major component in the research methodology for understanding human land relationships, even though this may not be sufficient in and of itself – studies in animal, cognition and robotics support this idea, and further research is warrented (Wilson & Golonka 2013). A biosemiotic framework that supports investigation of such spatial structures shall lead us to conclude that space is not “empty”, but is a product of semiosis (Ireland 2015).

Thus the unique sound of tree species when the wind moves their leaves (Haskell 2017; Schafer 1984) is a sonic object that can serve as an index in an organism’s Umwelt, in the same manner that cloud patterns allow humans to interpret and anticipate upcoming precipitations. Such ‘percepts’ can be categorized within a spatial ontology as an event depending on their spatio-temporal characteristics (Farina 2016).

Conventionally, matter and objects are considered ontologically prior to processes and events, but Galton and Mizoguchi (2009) challenge this traditional object-centered view that is reflected in the ontologies that have dominated Western thought. They do so by contrasting this view with an increasingly popular one that considers processes and events to be ontologically prior. Through its focus on relationships occurring between biological, physical, and psychic systems, the field of biosemiotics has contributed to the latter view by providing conceptual tools with which to understand change and emergent processes. However, Galton and Mizoguchi, suggest a third perspective according to which each of these pairs of categories is ontologically dependent on each other. They propose a “process-based criterion for object definition [that] makes no reference to boundaries, which is why it is able to pick out entities such as collectives through their behavior even though it may be impossible to circumscribe them by any meaningful boundaries” (Galton & Mizoguchi 2009 : 28). Even though they do not specifically address multisensory objects, I believe their position is well suited to support a multisensory based spatial ontology from an ecological perspective. Their description also strongly reminds us of the concept of Umwelt as articulated by von Uexküll and outlined by Favareau as a “set of agent-object relations reducible neither to the organization of the subject nor to the organization of the environment but always as the product of the interaction between the two, … and these relations of seemingly ‘private and subjective experiences’ are examinable …by science” (Favareau 2010 : 83).

Umwelt and Index (Biosemiotics and Space)

In von Uexküll’s conception of the world of meaning, each Umwelt forms a closed unit in itself, which is governed, in all its parts, by the meaning it has for the subject. “This space is built up by the animal’s sense organs, upon whose powers of resolution will depend the size and number of its possibilities for sensorimotor interaction” (Uexküll 2010 : 94), namely, the spatial perceptual structure. This applies to all organisms, including humans, and ideally signal interference is avoided in inter- and intra-species communication (Krause 1993). These worlds interact and overlap – bringing to mind the concept of semiosphere, first outlined by Lotman, a sphere in which it becomes possible to interpret messages in inter-species communication. A well-known illustration of this would be the communication through semiochemicals between insects and plants (Harborne 2001). In our bee example, it ultimately benefits humans if they do not disturb the path from hive to nectar through the atmosphere, but since we do not recognize this path as an object we may do so unwillingly. Therefore, it will be necessary to develop means of awareness of the signal(s) that creates this path (index), such as the polarized light and the aromatic volatiles released by the plant.

Within the human species, perception is furthermore conditioned by cultural codes. The use of symbols is associated with human cultures and communication, whereas most forms of animal communication are indexical. As argued by Deacon (2010), the human competence to interpret something symbolically depends on already having the competence to interpret many other subordinate relationships, indexicality, and so forth. But based on recent insights regarding the close relationship between spatial knowledge and the environment (Palmer 2015), we can reasonably assume that the mind is not independent but embedded within the body and its environment; indices provide us access to that world. The assumption that organisms, including humans, use indices to experience and know their world underlies biosemiotic research, for which I found support in my own research and practice. The spatial structure of perception can then be thought of as a multiplicity of triadic relationships within a geo-centered frame of reference.

According to Peirce, three categories are necessary and sufficient to account for all of human experience, corresponding to “firstness, secondness, and thirdness, whereby firstness is a conception of being independent of anything else (e.g. sensation), secondness is a mode of being that is in relation to something else, and thirdness is a mediator through which a first and second are brought into relation (Everaert-Desmedt 2011) In general this scheme takes into consideration emotional, practical, and intellectual experience. A sign can be simple or complex, any thing or phenomenon may be considered a sign the moment it enters into a process of semiosis involving a triadic relationship between a sign or representamen (first), an object (a second) and an interpretant (a third). A representamen in turn can refer to its object by virtue of firstness, secondness, or thirdness, that is, through relationships of similarity (icon), contextual contiguity (index) or law (symbol).

In Peirce’s words, an index is a sign or representation, which refers to its object because of it its dynamical (including spatial) connection both with the individual object on the one hand and with the sense of memory of the person for whom it serves as a sign on the other hand. Indices 1) have no significant resemblance to their objects, 2) refer to individuals, single units or single collection of units, or single continua, 3) direct attention to their object by blind compulsion (Peirce, CP 2.306 as reprinted in Favareau 2010 : 128). How then are we to think about indices within a geocentered framework, in order to assess our ability for survival based on sensory information in our surroundings? The structure or underlying logic is not species specific and applies to all modes of communication.

Clearly, most of today’s human population lives in urban settings that are highly artificial, yet our survival is dependent on the successful communication between organisms that co-inhabit this world and thrive in a natural setting. Even though the biosemiotic framework provides conceptual tools to transcend this nature-artifice divide, important aspects of bridging the divide have only been scantly addressed. Notably, coupled oscillations and synchronization of biorhythms is a known phenomenon in biological systems and the fact that being and moving in green (natural) environments is associated with improvement in human health conditions is assumed to be a result of such synchronization; this has recently gained more attention (Glass 2001; Strogatz & Stewart 1993).

It is not yet clear if “green” is only an index for other aspects (e.g. sound and odor signals) in these settings, and as such functions as a simplified symbol in our visual oriented world to assess the suitability of our surrounding for survival. What is even less known is why we do not get similar benefits from ‘synching’ with industrial rhythms and beats, but this may have something to do with the difference in dimensionality, the specific fractal dimension of natural phenomena (Bouzy 1995). This, I believe is a research topic that can and should be addressed within a biosemiotic framework. My current exploratory fieldwork in rural settings is a step in that direction.

“Greenness”, rather than a visual quality, is also an index used in environmental modeling to study climate change. It is calculated from remotely sensed and image processed satellite/aerial data. Simply said, through analyzing reflectance values, it indicates vegetation health of a specific land surface area during the time of data capture and as such provides us with information on changing environmental conditions on a global scale when monitored at temporal intervals. It is then symbolically represented in maps, but to understand such symbols requires a thorough understanding of the underlying references. Yet greenness can also be considered as a more poetic representation of the land, one that connects health and aesthetic experience and a gateway for a new direction toward spatial understanding, a rhythmic pulsing, in which green as a cyclic pattern can play its part (Glass 2001). Why is ‘seeing green’ so important to our health and aesthetic experience, as is demonstrated by an increasing number of studies (Brown et al. 2011)?

In general, these studies of ‘green’ focus on visual perception; in other words, research focuses on whether ‘seeing green’ makes us calmer, happier, etc. (Grinde et al. nd.) Elsewhere, we argue that “green”, as a strong (visual and intellectual) symbol of our healthy surroundings, impacts our ability to distinguish other indices of sustainable communication within and among organisms (van der Elst et al. 2018). We thereby build on studies of sensory integration that, among other things, assess the dominance of mode in perception, when diverging stimuli are presented (Stein & Meredith 1993).

Support for this idea comes from a decades long research in ecoacoustics (Farina 2016; Krause & Farina 2016). Krause (1993) in particular shows that change in environmental conditions can be detected via soundscape analysis prior to analysis from satellite imagery (visual domain). The persistence of ‘green’ as a symbol of healthy environment can easily obscure other indexical relationships that may be more informative about the state of health of our Umwelt. Clearly, the physical environment/ambient sphere is not visual only, but exists of interrelated constellation of sensory signals in different sensory domains (Hope 2010). In his case, Krause shows that the sonic environment provides a better index of the nature of change. It is not unlikely, that many other referential relationships can increase the resolution and dimensionality of an indexical network within human range.

Even though human behavior and intention has long been focused on controlling their environment, as a species we are equally tethered to other species and dependent on these relationships for our survival. Only recently can we demonstrate that these other species can sense and perceive the sign languages of others (Harborne 2001), as a spatio-temporal choreographed chemical sphere, probably not unlike the acoustic space of ‘The Great Animal Orchestra”, revealed to us sonically in a new way in the 20th century by Krause (2012). Outlined initially as the niche hypothesis, Krause shows us that each species uses a unique bandwidth to communicate its message, to minimize noise and interference. Human activity over the last century or so has changed this communicative sphere dramatically, which also has resulted in many unforseen consequences. Just like the human impact on soundscape, our ignorance of the complexity of multimodal communication has already led to dramatic impact on chemical signaling as well, in more than one way (McFrederick 2009; van der Elst 2016).

My current fieldwork includes prolonged periods in rural areas to gain insights in the variables in human-land relationships; one of these networks is the one managed by beekeepers. Bees are wild and nomadic, they are not domesticated animals, but beekeeping has developed as a form of husbandry that allows humans to parasitize on bee food production. It is through this activity that we have gained insight in the (spatial) intelligence of the bee community. Unfortunately, health conditions among bee populations are deteriorating rapidly and bees are dying at alarming rates. Now, beekeepers are becoming crucial observers of changing conditions in the semiotic network. Certain chemicals used in industrial agriculture are directly responsible for the most dramatic, rapid decline. These are not however the only factor, as flowers are essential.

The greenness index to assess vegetation health as described above is part of an environmental monitoring suite. Through processing of satellite data, it is also possible to identify the type and the condition of the plants that make up this green. Each species in fact reflects and absorbs the Electro Magnetic (EM) radiation in its own way, resulting in unique EM spectral signatures. The ‘object’ in this case is not ‘the plant’, but ‘the plant as it goes through different phases’, an interplay between its internal processes and external processes. For instance, the acacia that is in bloom during late April, early May in the mid northern latitudes can be identified in satellite images. In 2017 a cold spell in the spring negatively affected vegetation in the northern and central parts of Italy, subsequent heat and drought later in that same spring further exacerbated the situation. The acacia was still blooming, but something was missing, the scent is was faint. Several beekeepers understood what this meant : there would be no or little acacia honey since there was no nectar in the flowers, and bees would have to forage on other flowers if they find them. So even though satellite images show acacia in bloom, the picture is deceptive as it is missing its essential component. While bees operate in the visual domain to help them navigate through the atmosphere to locate the flower’s “landing pad”, they are highly dependent on their olfactory skills to identify source locations – in other words, to sense out different flowers to replace acacia (Chittka & Brockmann 2005; von Frish 1970). The beekeepers’ role consists in expanding this network, by transporting the bees to appropriate feeding locations. This “ground-truthing” is a skill performed by many small scale-farmers, whose knowledge of indexical relationships is a valuable resource.

In general, spending time in rural settings makes one more aware of these different modal objects. Personally, this experience is also informed by my research in anthropology and spatial sciences, something that helps me recognize that ontologies of space are multimodal, that they come in different configurations, and that they become frames of reference for the different networks with which humans engage. Previous experience with agricultural communities, where I integrated geospatial technologies with culturally appropriate approaches, made me realize that I didn’t fully understand various aspects of human-land relationships, something I developed through experiential work. Thus, over the years I have been especially interested in food relationships that are in many ways traditional and require manual labor : collecting seaweed in Okinawa, gathering walnuts in Hungary, wine making, honey, olive oil and saffron production and truffle hunting in Italy. Participating in these activities enlightened me to the necessities of in-depth knowledge of the land, the importance of a close human-land relationship, and the need for indices that structure these semiospheric activities. Subsequently, this work also made it clear to me that many scientific studies lack this kind of prolonged investigation in ecological relevant settings. Especially in today’s world, the human participatory role as well as our dependent position is often underestimated.

Understanding the world of animal communication as advocated by von Uexküll makes our world effectively bigger. Furthermore, it opens up the possibility to reassess human spatial intelligence with the potential to share experiential worlds in new ways and guide developments in emerging spatial technologies.

Boundaries & Embodied Approach (Navigating the Semiosphere)

Starting from the premise that the senses are a primary source of knowledge, it seems obvious that cognitive development is rooted in the sensory-motor system and in bodily movement. However, according to Smith and Sheya (2010), cognitive science developed from the idea that cognition is separate from sensory-motor processes and only recently arrived at the idea that cognition stems from the body/bodily experience. A link thus exists between the body, its actions, and spatial cognition. While the specific cognitive mechanisms underlying this link are still under debate, the fact that the body plays an active role in the cognition of space is, at this point, generally accepted and many research efforts have been dedicated to understanding how sensory information about the body and the environment is converted into action. Starting from this premise Clifton et al. (2016) discuss two main approaches that have developed from this research for understanding spatial and other cognitive processes : ideomotor theory and ecological-based embodied cognition. In their work on Tangible Embodied Interfaces (TEI), Clifton et al. pursue an ideomotor approach, rooted in spatial information theory. Ideomotor theory, which has emerged out of information processing, is described through the representational accounts of the series of events that occur from sensation, through perception and cognition, to action, yet are reciprocally connected in such a way that action and bodily systems can prime or shape perceptual representations and processes. In contrast to the ideomotor approach, an embodied approach to cognition rejects the notion of representation; action possibilities or affordances (Gibson 1979) are directly perceived from the environment. Perception and other cognitive events are then embedded firmly in the body and the environment in which the body is acting – this idea is supported by recent findings in spatial thinking (Palmer 2015).

The notion of rejection of representation makes an ecological approach the preferred one for understanding biosemiotic relationships. As argued by Pickering :

Embodied action, not language, is both the ontogenetic and phylogenetic origin of mental life. Language and mental representation are neither phylogenetically typical nor developmentally fundamental. This proposition, that experience does not require representation, is central to embodied cognition, which opposes the implicit anthropocentrism of the cogito.

2005 : 198

Even if language provides us with a means to gain insight into the nature of spatial experience, it is not necessary for developing spatial skills.

Empirical research by Hegarty et al. (2006) further supports this idea. Their study compared movement of three groups in different environments in order to assess their spatial navigation abilities. Their results showed that people who moved through a physical/natural environment were better able to make judgments about the distances and directions between points of interest than people who navigated in a virtual environment, or watched a video of the environment being navigated. This research however can be considered unique in its focus since, according to Bouzy (1995), many works in spatial reasoning deal with artificial objects instead of natural objects; hence they do not readily address how we categorize and reason in landscape settings. This is major gap in our understanding of spatial experience. The problem, Bouzy argues, is that

the structure of natural landscapes strongly differs from artificial ones, the first intuitive difference is regularity and becomes more complex when we start to consider (fractal) dimensionality of natural objects. Boundaries are significantly different in artificial environments than in natural settings. (1995 : 1)

Galton and Mizoguchi address this issue from a logical point of view. They use the example of a waterfall to demonstrate how to identify an object as an interface between processes which are internal to it and those which are external to it and which it may be said to enact. To describe a waterfall we are faced with mutually inconsistent options, either we describe what it is, or else what it does; the former is a configuration of falling water or the transfer of water from a higher to lower elevation (conduit). But when the waterfall is observed for a longer time, the waterfall migrates upstream by carving into the rock and can be considered “as a ‘device’ for moving the rocky precipice upstream along the river channel, through the agency of the falling of water, which constitutes its primary internal process” (2009 : 17). Thereby the objects are linked intrinsically to the process in which they are involved. In this view, neither matter/object nor process/event is ontologically prior to the other : both are interdependent.

When we think of a boundary in the broadest sense, we think of an entity (or event) demarcated from its surroundings by a genuine material discontinuity forming its surface or boundary (Smith 1996). However, Galton’s and Mizoguchi’s process-driven criterion makes no reference to boundaries. In this manner, it opens up a new way to integrate visually based objects with, for instance, sound and odor objects, since entities such as collectives can be identified through their behavior. It follows that through its very connectedness an object is, in fact, endless. An object is never described in its entirety, but can be considered from different points of view (Galton & Mizoguchi 2009 : 29). Again, the similarity to von Uexküll’s ideas come to mind. Objects play a role in different but overlapping Umwelts and as such can be studied using this ontological view.

Enter odor. Bees have a visual sense organ, but are highly dependent on recognizing scented sources. In some cases, they must obtain a description of the spatial location and the individual odor quality of environmental odor sources through olfaction alone. The bee’s world consists predominantly of odor objects and the ability to memorize and recognize such objects in their surrounding.

The variable nature of turbulent air flow, through which these objects are mediated, makes such a remote sensing problem solvable if the animal can make use of the information conveyed by the fluctuation with time of the mixture of odor sources. Behavioral evidence suggests that such analysis takes place.

Hopfield 1991 : 6462

Since space is represented from a human perspective mainly as visual, olfactory psychophysics and electrophysiology studies are usually concerned only with a single odor presented to the nose (Turin 1996) and ignore the function of olfaction of other species which help define and locate specific smells in a complex environment (Hopfield 1991). Hopfield furthermore argues that the olfactory system is different from the visual one, mainly because the problems to be solved from a remote sensing perspective are different. An animal uses scents brought by the wind to identify the direction of and approximate distance to odor sources, but in order to do so it must perform certain tasks : odor memory and recognition, background elimination, component separation, and odor separation. We can think of this in terms of odor ontology and aromatic indices, comparable to the visual perception indices created by Higuchi (1983).

Researchers like Schafer, Krause and others have brought the importance of the soundscape to our attention; nonetheless, the role of semiochemical indices in our environment is still underappreciated. However, awareness of interference in this domain is growing since pollution impacts the atmosphere at global scale. Aromatic volatiles released by flowers to attract pollinators dissipate quickly, transformed by pollutants and thereby impeding the insect’s ability to perform its tasks for successful spatial navigation and the risk that no pollination will occur. As mentioned earlier, the impact of air pollution on the bee’s olfactory map appears to be dramatic and it is hard to imagine how meaningful odor fields, paths and corridors in these species’ Umwelten have changed. If you can imagine, this is a very different Umwelt from the human visual space. Yet as humans we interact in it.

Odors of the land and semiochemical communication are a focus within the Forest Bathing project, a project I initially started as part of the Machine Wilderness platform (www.machinewilderness.net). In collaboration with other artists and scientists, I wish to better understand the interrelationship between ‘aromatope’ and ‘sonotope’ (Krause & Farina 2016) and eventually other sensory fields. What is at stake, in other words, is the interrelationship between acoustic and olfactory sign communities within an ecology of senses through experimentation with new technologies, and by developing novel methodologies, in which these technologies can become part of a sustainable semiosphere. Since humans are unable to recognize many sensory objects in the semiosphere, proxies or indices can provide means to tap into this communicative network. The present discussion serves as a stepping-stone to further work, some of which I continue in “Surfing the Semiosphere” (van der Elst, forthcoming).

Although research on odor ecology can still be considered as being in its early stages, it is not unlikely that a similar ‘niche’ hypothesis (sensu Krause) underlies olfactory communication, a bouquet of the land in tune with its animal orchestra, in which rhythmic variability is an important, but not the only, object characteristic. In order to develop novel technologies to better understand these (inter)relationships, it is necessary to outline an equally novel framework that is not rooted in the machine (computer) metaphor, (in other words, organisms don’t compute their environment), but rooted in biology. What we need is a framework to model semiotic relationships across subjective experiential worlds and technologies or interfaces (sensu Galton & Mizoguchi) to help us connect processes and identify the indices that give new meaning to these processes. This is what I set out to explore; the research is in its initial stages, but nonetheless pointing to an exciting journey. Technology plays an important but subordinate role in this endeavor. The objective is to enhance awareness and maintain sensory richness (akin to species richness).

Case in point : I have adopted an unconventional method to gain an understanding of what embodied research entails, beyond a phenomenological (or ego-centered) approach. As mentioned above, starting from the premise that natural environments are native human habitats, I have spent much time working with organic and biodynamic farmers in remote rural settings to gain insight in human-land relationships and knowledge of the land. The idea is that in such settings the external processes, e.g. atmospheric conditions, are likely closer to optimal for living than for instance urban settings. Being immersed in the rhythms of the land, the concepts of Umwelt, indexicality, and interface as a framework for engaging with the world begins to make much more sense and takes on new meaning.

Groundtruthing

The implication of the need for an embodied approach is tremendous. While laboratory studies are still valid and necessary in scientific research, I believe we need a new methodology that includes prolonged fieldwork studies in ecologically relevant and embodied contexts. This is because many ecological processes cannot be mimicked in laboratory settings (Splivallo et al. 2001). This methodology assumes that humans are interrelated subjective observers, not outsiders. The field is thus envisioned as an ‘outdoor immersive environment’, combining elements of ethology and cognition with developments in emerging spatial technologies.

The concept of ‘groundtruthing’ is illustrative. It commonly describes a component of image analysis of satellite data as an earth observing system. Analyzing satellite imagery is complex, as it is not always clear what the patterns or specific spectral signatures are represented by the imagery. Therefore, it is necessary to check the ‘pixel’ on the ground.

Imagine the acacia tree stand; within its spectral signature it will show flowering around the month of May in northern latitudes in the greenness index. For the bee families the white color of the flower will guide them, but something else will tell them whether or not they will find nectar. On any given year the flowers may bloom, but very limited nectar is available; the message ‘no nectar’ cannot be deduced from the satellite imagery. You have to be on the ground, you have to listen… no buzzing bees, and smell…only a faint acacia aroma. We need to develop many more indices, tapping into the sensory richness of our surrounding world. Developing new tools and technologies can help us gain access to aspects of the world we currently ignore; groundtruthing can help to develop sensitivity and awareness of the delicate world in which we participate.

Concluding Remarks

This paper started from the idea that the way we organize our worlds is based on boundaries and through explorations of the languages we use to express those boundaries we can gain insight as to whether these boundaries represent the world or just the workings of our minds. Formulating the issue in this fashion we are reminded of the age-old problem of the mind/body dichotomy that underpins Western science.

Research in the field of biosemiotics proposes an alternative approach. Based on the work of Peirce who considered all forms of thought (ideas) essentially as communication (transmission of signs), biosemiotics is rooted in the idea that meaningful communication takes place within and among organisms, a model of the world put forth in von Uexküll’s concept of Umwelt. Exactly how these different subjective worlds are spatially configured and interrelated is not well understood, but the idea is essential for moving forward with this research.

Several ideas and recent insights in human spatial experience were introduced in this paper, for example, studies that show that people who navigate in physical environments can better judge spatial configurations than those who navigate virtual spaces, and studies of people who show improved health conditions (measured with biomarkers such as bloodpressure) after spending time in green environments. Given that spatial cognition is a fundamental cognitive domain, these ideas point to embodiment as central to experience and spatial knowledge. Such a proposition also underpins the research and development of Tangible Embodied Interfaces (TEI) by Clifton et al. (2016) yet theirs is still dominantly grounded in research in the visual domain that largely neglects multisensory objects in a human (organism) Umwelt. The discussion in this paper focused on new findings in spatial ontologies and reference systems to explore the boundaries of these Umwelten, especially objects that are not visually based. Sound and odor objects are only the beginning of ‘imagining’ a sensory-based world relative to the totality of sensing organs with which organisms are equipped, but they already provide an exciting starting point. What is exactly an object in such a sensory-rich world, what are its boundaries? The beauty of the object definition proposed by Galton and Mizuguchi is that it makes no reference to boundaries; the process-driven approach allows objects to be inter-related through connecting internal and external processes. One thing points to another.

Tentatively we could claim that the ability to understand indexical referential relationships is fundamental in spatial experience, based on arguments by others that representation – as a unique human cognitive ability – is not required for understanding space. Taking this even further we can question whether the focus on (visual) representation hinders the development of human spatial skills. Visual/spatial thinking is a dominant mode in the Western scientific knowledge system that constitutes the symbolic knowledge framework – together with linear, sequential thinking. Yet does the ability to manipulate and depend on symbols negate our ability to understand the underlying indexical referential relationships, instead of assuming that this ability infers an understanding of ‘subordinate’ (e.g. indexical) relationships? Further research is needed. Addressing these questions is beyond the scope of the current paper, but the brief discussion of spatial aspects related to biosemiotics will hopefully serve to spur new research directions and cross-over practices in human-land and interspecies relationships.