Theme sessions

Engaging with the Senses: Genres of Synesthesia and the Scrambled Senses of (Some) Other Creatures

Conveners:  Myrdene Anderson (Purdue University) and Donna E. West (State University of New York [Cortland])                            

While human senses and emotions are classified for us by the various linguïcultures, the former are finite while emotions are more nebulous despite being profusely labeled.  After sight and hearing, the lesser-specified and more-focused senses in English— olfaction, taste, and touch—carry even more weight in their cognitive associations with emotion, consciousness, and memory.

The human sensorium seems orderly only when referred back to the loci associated with perception of external stimuli, this process heavily shaped by cognitive expectations, and cognition further inflected by particular linguicultural habits.  However, hominization demoted some senses as our ancestors became bipedal with forward faces and stereoscopic vision.  Although sighted humans can declare with a shrug that “seeing is believing”, some synesthetes perceive with embellished features, or with commingled other sense modalities.  Consistently seeing black letters as colored is a common condition among synesthetes.  Less common would be experiencing sound alongside sight (perhaps logical enough in literate societies with habits of reading aloud) or also experiencing odor, flavor, or contact.  However, no individual can assume that the sensations, perceptions, or cognitions are comparable, whether or not they have earned semantic labels, and individuals experiencing multi-sensory integration may be unaware that it's unusual.

Senses beyond the basic five (or six) include balance, temperature, time, movement, and proprioception.  Venturing beyond the outside-body monitoring tied to sense organs and the more ephemeral ambient and body-internal sensing that can show signs of contagion across bodies, one swerves toward full-blown yet culturally-shaped emotions themselves, implicating particular underlying cognitions.  Here, waves for light, sound, and aromatics combine with other media connecting outsides with insides—seeking to establish some of the quantum entanglements or cross-modal conceptions of sensation, perception, kinds/degrees of awareness/consciousness, mindfulness, feeling, will, emotion, memory, empathy, synesthesia!  Or might inter-modal experiences be explained in terms of Peirce’s notion of dreams as verbs, his creative hallucinations (1903: EP2: 192), or his coinage of virtual habit (1909: MS 620)?  Peirce characterizes dreams as verbs, not merely materializing in vocal production-to-ear, but vocalization-to-action schema.  In terms of hallucinations, Peirce presents three kinds: obsessional, social, and creative.  It is primarily the latter which most demonstrates synesthesia-based experiences (while gleaning truth value), i.e., translating an image in the mind into a viable hunch /inference, such that an imagined touch feels so physical that it is deemed to be so.

This theme session will address the cross-sense modality transfers taken in stride by human synesthetes, together with the question: can empirical research ever adequately measure/characterize the human experience of seeing music, tasting touch, smelling color, let alone unbraid the deduced sensoria of other animal species?  For instance, there is now evidence that, in the canine sensorium, olfaction does not stand alone but inflects and infects the other senses, including sight (Andrews et al.  2022).  That might translate into the following inter-modal skills: dog vision would be in odor as much as in color; bird speech would be in tempo with kinesthetic rhythm; and snakes may smell with the tongue.  And then, there be plants (cf. Gagliano 2018).

References

Andrews, Erica, Raluca Pascalau, Alexandra Horowitz, Gillian M. Lawrence, and Phillippa J. Johnson.  (2022) Extensive connections of the canine olfactory pathway revealed by  tractography and dissection.  Journal of Neuroscience 42(33): 6392–6407.  https://doi.org/10.1523/JNEUROSCI.2355-21.2022

Cytowic, Richard E.  (2018)  Synesthesia (The MIT Press Essential Knowledge series). Cambridge: The MIT Press.

Day, Sean A.  (2022)  Synesthesia and synesthetes. Independently published; ISBN 979-8849880150.

Gagliano, Monica, with Suzanne Simard.  (2018)  Thus spoke the plant: A remarkable journey of  groundbreaking scientific discoveries and personal encounters with plants.  Berkeley: North Atlantic Books.

Howes, David, editor  (2009)  The sixth sense reader.  London: Routledge.

Jarvis, Erich  (2019)  Evolution of vocal learning and spoken language.  Science   366[6461]: 50- 54.  doi: 10.1126/science.aax0287

Ullmann, Stephen  (1964)  Language and style: Collected papers of Stephen Ullmann.  Oxford:   Basil Blackwell.

Yong, Ed  (2022)  An immense world: How animal senses reveal the hidden dimensions around us.  New York: Penguin

Linguistic Bodies in Interaction

Conveners: Šárka Kadavá (Leibniz-Centre General Linguistics, Berlin) and Wim Pouw, (Donders Centre for Cognition, Nijmegen)

In communication we use a myriad of time-varying signals – besides speaking or signing – to create and sustain mutual understanding. These dynamic signals include gestures, facial expressions, eye gaze, or any posture in motion that can be recruited for intentional communicative signaling. The language sciences have long overlooked the polysemiotic and intersubjective context of these signals, directly contributing to a crisis of meaning by hindering our understanding of the nature of communication. This crisis is rooted in two interrelated issues: (1) an overly individualistic approach, and (2) a monosemiotic, static language-centered perspective. Addressing these issues can help us acknowledge that communication is, in fact, fundamentally intersubjective and dynamic.

Firstly, to achieve a more holistic view of language and how we negotiate meaning, we need to shift our focus beyond ‘single-mindedness’ (Dingemanse et al. 2023). Despite the methodological challenges this shift may pose, it is crucial to enrich isolated experimental settings with more ecological dyadic interactions that facilitate the full range of dynamic signals used in face-to-face communication. Secondly, the challenge of dynamicity should be overcome. For analytic purposes, one often needs to identify and capture complex phenomena with relational interdependencies as if decomposable in independent functional variables. As far as language is concerned, this means abstracting from time-varying materially constituted informational flow into static, second-order patterns (e.g., words, norms, etc.). However, those patterns are incomplete in capturing ‘moving matter’ and ‘movement that matters’ and reduce the complexity of the communicative act.

Recent research has increasingly recognized this missing link, advancing the field both methodologically and theoretically. Since the ‘multimodal turn’, researchers have started to appreciate the orchestral nature of body articulators and their neat temporal synchronization (e.g., Rochet-Capellan et al. 2008), and the physiological properties constraining communication (e.g., Pouw et al. 2020). Finally, more evidence is brought forward regarding the variety of communicative and/or pragmatic functions that motion – such as pointing (e.g., Raghavan et al. 2023), or eyebrow movement (e.g., Nota et al. 2023) – can take on in the interaction, making use of different dynamic patterns of our bodies (e.g., acceleration, amplitude, etc.). Moreover, cognitive semiotics can valuably contribute to the discussion as it offers a transdisciplinary approach that instead of focusing on language per se highlights the polysemiotic nature of meaning making (e.g., Zlatev, 2015; Zlatev et al., 2023).

This theme session aims to bring together researchers who explore how dynamic signals serve as a means of relating to each other and enlighten the ways in which we overcome everyday crises of meaning. Speakers will approach this theme from a variety of perspectives – e.g., pedagogical, technological, and developmental – and cover topics such as co-singing gesture interaction, video mediation of linguistic bodies, or repair in novel communication. The session provides an opportunity to bring together researchers at different stages of their academic careers, working under different theoretical frameworks aligned with cognitive semiotics, focusing on various target groups, etc. This is relevant to prevent and/or resolve our scientific crisis of meaning and to re-think how meaning and meaningful signals unfold when we interact. Among other things, it is exactly those dynamic signals that can bridge the gap between individual minds and frame language and communication as ‘something we do together’ (Fusaroli et al. 2014).

References

Dingemanse, M., Liesenfeld, A., Rasenberg, M., Albert, S., Ameka, F. K., Birhane, A., ... & Wiltschko, M. (2023). Beyond single‐mindedness: A figure‐ground reversal for the cognitive sciences. Cognitive science, 47(1), e13230.

Fusaroli, R., Rączaszek-Leonardi, J., & Tylén, K. (2014). Dialog as interpersonal synergy. New Ideas in Psychology, 32, 147-157.

Nota, N., Trujillo, J. P., & Holler, J. (2023). Specific facial signals associate with categories of social actions conveyed through questions. PloS One, 18(7), e0288104.

Pouw, W., Harrison, S. J., & Dixon, J. A. (2020). Gesture–speech physics: The biomechanical basis for the emergence of gesture–speech synchrony. Journal of Experimental Psychology: General, 149(2), 391.

Raghavan, R., Raviv, L., & Peeters, D. (2023). What's your point? Insights from virtual reality on the relation between intention and action in the production of pointing gestures. Cognition, 240, 105581.

Rochet-Capellan, A., Laboissiere, R., Galvan, A. & Schwartz, J. “The speech focus position effect on jaw–finger coordination in a pointing task”. J. Speech Lang. Hear. Res. 51.6 (2008), pp. 1507–1521.

Zlatev, J. (2015). Cognitive semiotics. International handbook of semiotics, 1043-1067.

Zlatev, J., Devylder, S., Defina, R., Moskaluk, K., & Andersen, L. B. (2023). Analyzing polysemiosis: language, gesture, and depiction in two cultural practices with sand drawing. Semiotica, 253: 81–116https://doi.org/10.1515/sem-2022-0102

Reigning in Neuromania through Cognitive Semiotics

Conveners: Filomena Diodato and Alice Orrù (Sapienza, University of Rome)

“Neuromania” is a term used for over-enthusiastic but simplistic appeals to neuroscience in the humanities and social sciences (Tallis, 2011; Legrenzi & Umiltà, 2009/2011). Widespread in the humanities, from semiotics to politics, current “neuromaniac imperialism” (Tallis, 2011: 73) both trivializes the undisputed cognitive role of the brain, and falls prey to naive neo-behaviorist and materialist conflations of body, brain, and mind. It risks both rehabilitating pseudosciences like phrenology and downplaying forms of consciousness that define human beings in their species-specificity.

Neuromania is also found in quarters of cognitive linguistics, a close ally of ideas such as the cognitive unconscious (Lakoff & Jonhson, 1999). In a “cerebrocentric perspective” (Pennisi & Falzone, 2016), cultural artifacts like signs and situated and sedimented practices are ignored or traced back to mere outcomes of ancestral survival experiences, disregarding the equally formative role of history, culture, and communicative practices (Donald, 1991; Deacon, 1997; Logan, 2007). Neuroscience may indeed shed light on the mystery of the human mind, but it can arguably do so best when accompanied by a cognitive-semiotic approach, suited to the investigation of sense-making processes (Sonesson, 2018), directed at explaining how shared semantic fields emerge from an infinite variety of intersubjective semiotic games. This is, after all, the space of freedom in which human beings build, based on their bio-physical endowments, a Lifeworld not reducible to the physical realm of matter.

With the aim of understanding both the pitfalls of neuromania, and the potentials of neuroscience (e.g. Sinigaglia & Rizzolati, 2022) this theme session turns to phenomenologically grounded cognitive semiotics (Zlatev, 2012) and some non-reductionist enactivist approaches (Diodato, 2020). In the presentations, we aim to show how cognitive semiotics can both “reign in” neuromania, and/or rehabilitate non-reductionist neuroscience. 

References

Deacon, T. (1997), The Symbolic Species: The Co-evolution of Language and the Brain, W. W. Norton & Company.

Diodato, F. (2020), «The embodiment of language: sign function and semiotic threshold», in RIFL SFL: Language and Emotions, pp. 203-215.

Donald, M. (1991), Origins of the modern mind: Three stages in the evolution of culture and cognition, Harvard University Press.

Lakoff, G.; Johnson, M. (1999), Philosophy in the flesh, Cambridge University Press.

Legrenzi, P.; Umiltà, C. (2009), Neuro-mania. Il cervello non spiega chi siamo, Bologna, Il Mulino (Engl. Transl. 2011, Neuromania: On the limits of brain science, Oxford Academic)

Logan, R. K. (2007), The Extended Mind: The Emergence of Language, the Human Mind, and Culture, University of Toronto Press.

Pennisi, A.; Falzone, A. (2016), Darwinian Biolinguistics. Theory and History of a Naturalistic Philosophy of Language and Pragmatics, Springer.

Sinigaglia, C; Rizzolatti, G. (2022), Mirroring Brains: How we understand others from the inside, Oxford University Press.

​​Sonesson, G. (2018), New Reflections On The Problem(S) Of Relevance(S). The Return Of The Phenomena: Theories, Factors and Challenges, in Strassheim, Jan and Nasu, Hisashi, 2018, Relevance and Irrelevance: Theories, Factors and Challenges, De Gruyter, pp. 21-50.

Tallis, R. (2011), Aping Mankind. Neuromania, Darwinitis and the misrepresentation of humanity, Acumen.

Violi, P. (2008), «Beyond the body: Towards a full embodied semiosis», inR. M. Frank – R. Dirven - T. Ziemke – E. Bernárdez (eds.), Body, Language and Mind, Vol. 2: Sociocultural Situatedness, Mouton de Gruyter, pp. 53-76.

Zlatev, J. (2012), «Cognitive semiotics: An emerging field for the transdisciplinary study of meaning», Public Journal of Semiotics, 4(1), pp. 2-24.

Sonesson and phenomenological cognitive semiotics

Conveners: Jordan Zlatev and Alexandra Mouratidou (Cognitive Semiotics, Lund University)

As it exists today, cognitive semiotics would have been impossible without the key contributions of Göran Sonesson, who sadly left us in 2023. Arguably his major achievement in this respect has been to help place cognitive semiotics on the firm basis of phenomenology, the philosophy (and psychology) of the lifeworld and the way it is given in human experience, inaugurated by Husserl in the beginning of the previous century. In Sonesson (2009) this is emphasized explicitly, but it is evident almost everywhere in his work, starting from the now classical Pictorial Concepts (Sonesson 1989).

Among the many key ideas of phenomenological cognitive semiotics emphasized by Sonesson have been the following. First, the foundation of human meaning making is to be found not in signs but in perception, which along with imagination should be understood not as based on “mental pictures” or any other kinds of representations - but on enactive processes, such as those proposed in ecological psychology (Gibson 1979). On this basis, Sonesson (e.g. 2007) argued for a narrower definition of the sign than customary in Peircean semiotics (but broader than in Saussurean semiotics), namely as a derivative form of intentionality based on reflective consciousness, necessary for both relating and differentiating between expressions (representamina) and intentional objects.

A special focus throughout Sonesson’s life has been the pictorial sign, where he built on Husserl’s analysis of pictorial consciousness, and the concepts of “picture thing”, “picture object” and “picture subject”, and developed the analysis by distinguishing the latter from the referent (Sonesson 2010).

Further, Sonesson contributed to developing the model known as the Semiotic Hierarchy (Zlatev and Konderak 2023), by arguing for a level of culture that does not presuppose sign use, and a level that does (Sonesson 2015) as well as the key methodological principle of Phenomenological Triangulation, by distinguishing the usual epistemological (vertical) axis of first-person, second-person and third-person perspectives, from an ontological (lateral) axis concerning the nature of the phenomena themselves: self, other and the world (Sonesson 2022).

In this theme session we invite contributions that explicitly build upon these (or possibly related) aspects of phenomenological cognitive semiotics, in order to both honor Sonesson’s memory, and to continue his work.

References

Sonesson, G. (1989), Pictorial Concepts: Inquiries into the Semiotic Heritage and its Relevance for the Analysis of the Visual World, Lund: Lund University Press.

Sonesson, G. (2007), ‘From the Meaning of Embodiment to the Embodiment of Meaning: A study in Phenomenological Semiotics’, in T. Ziemke, J. Zlatev, and R. Frank (eds), Body, Language and Mind. Vol 1: Embodiment, 85–128, Berlin: Mouton de Gruyter.

Sonesson, G. (2009), ‘View from Husserl’s Lectern’, Cybernetics & Human Knowing,16(3–4), 107–48.

Sonesson, G. (2010), ‘From Mimicry to Mime by Way of Mimesis: Reflections on a General Theory of Iconicity’, Sign Systems Studies, 38 (1–4), 18–65.

Sonesson, G. (2015), ‘Le jeu de la valeur et du sens’, in A. Biglari (ed), Valeurs: Aux fondements de la sémiotique, 85–101, Paris: L'Harmattian.

Sonesson, G. (2022). ‘Cognitive Science and Semiotics’. In J. Pelkey (Ed.), Bloomsbury semiotics Volume 4: Semiotic movements (pp. 293-312). London: Bloomsbury Academic.

Zlatev, J., & Konderak, P. (2023). ‘Consciousness and Semiosis’. In J. Pelkey (Ed.) Bloomsbury Semiotics Volume 1: History and Semiosis. Bloomsbury Academic.

Page Manager: iacs5sol.luse | 2024-01-13