Plenary speakers — Abstracts
Anne Beyaert-Geslin: The mediating photograph
A picture never comes alone but always proceeds from other pictures. The concept of remediation (Bolter & Grusin 2000) reflects a change of medium, for example from a photograph to a painting but does not indicate how a picture is informed by others. The difference that the english language makes between a picture or a material image and the image, a mental image, proves to be precious here and makes it possible to grasp this imaginary dimension, evoked on many occasions even though under various names (openness, participation, emptiness ... ) and by different approaches (Eco 1997, Basso Fossali 2017 ; Metz 1971 ; Neyrat 2003). Going to the limits of the manifestation, thus at the limits of its field of expertise, visual semiotics can explore the passage from the visible to the visual. It is this comparison that we wish to make by assuming that not only does the picture always proceed from other images, but that it is largely informed by photography (Poivert 2013) which is a meta image. If hyperrealism has exemplified this photographic perception of the world, the works of painters such as G. Richter or L. Tuysmans are also exemplary. But is the influence of photography not deeper and could it not be verified for other pictures and other media than painting? Benjamin (2000 [1931]) and Sontag (2008 [1977]) have previously emphasized this taking possession of the world by photography. By a corpus study, we will try to show this photographic formatting.
Johanna Drucker: How We Can’t Think/See: Lifelines and Timelines
The development of conventions for interface, visualization, and graphical layouts in web-based browser environments has occurred rapidly. But what aspects of these conventions limit the possibilities for expression? Looking into the history of graphical formats, we see many ways of forming information and language that have been eliminated in digital environments (the flexibility of manuscript production, for instance, allowed interlineation, commentary, and highly articulate interactions between various sections of a text that are very difficult to perform in digital texts). Information visualizations are equally constrained by a combination of conceptual and technical limitations that become habits of thought. Taking the example of the lifetime-timeline, this talk asks how we can’t think with current conventions—and how we might think differently in graphical forms that break with these conventions. Underpinning this argument is a sometimes unacknowledged connection between cognition and vision through which the visual aspects of language and enunciative features of visualizations can be understood.
Jennifer Green: Polysemiotic dimensions of Central Australian sand stories
Indigenous sand stories from Central Australia are an aesthetically rich exemplar of a multimodal, or ‘polysemiotic’ verbal art form (Green, 2014; Munn, 1973). A small set of semi-conventionalized graphic symbols are embedded in a complex semiotic field that includes sign, gesture, speech and song. Sand stories begin with the clearing of a space on the ground in front of the narrator. The resultant drawings and mini-installations of objects are both product and process and involve a complex interplay between dynamic and static elements. Some graphic traces depict memories of movement, re-enacting ancestral pathways or representing everyday journeys (Ingold, 2007). Others, such as the ubiquitous ‘U’ shape, represent persons, and yet other graphic forms illustrate the artefacts and accessories of everyday life. Between ‘scenes’ or ‘episodes’ the narrator wipes the space on the ground in front of them clean before beginning to draw again. The visible symbols drawn on the ground are thus designed to be perceived in the fleeting moment, rather than “stored for contemplation at a later time” (Hinkson, 2014, p. 10). Bringing a cognitive perspective to understanding how sand stories work raises methodological and conceptual challenges. At the micro-analytic level, disassembling sand stories into a series of coherent action units or ‘moves’ calls into question pre-conceived notions of what ‘drawing’ or ‘gesture’ might be. Some actions leave visible traces. When it comes to audible aspects of the stories, parallel challenges arise in determining the boundaries between speech and song. Using an approach that does not sequester communicative modes from each other (Murphy, 2012, p. 1966) but rather seeks to understand their integration, I draw on examples of sand stories from the Central and Western Deserts of Australia.
Antonis Iliopoulos: Cognitive Archaeology meets Visual Semiotics: Tracing the co-evolution of symbolism and selfhood through early body ornamentation
Cognitive archaeology, or the archaeology of mind (as it is also called), is a subdiscipline of archaeology that seeks to account for the origins of “modern” human cognition. While cognitive “modernity” has been linked to a broad range of technological and social abilities, it has been most eminently tied to the capacity for symbolism. Many evolutionary cognitive archaeologists have thus directed their attention towards early body ornaments, which they consider as undisputable evidence for symbolic behaviour. To give but a telling example, shell beads such as those found at Blombos Cave are widely accepted as conclusive evidence that a capacity for symbolism and, by extension, modernity had evolved in South Africa by about 75 thousand years ago.
In making these claims though, evolutionary cognitive archaeologists seem to rely on a pair of misguided assumptions: (i) that material signs are inherently symbolic because they are all arbitrarily linked to their meaning; and (ii) that the use of symbolic culture implies human beings with a concrete sense of self, which employed material signs in order to communicate personal information to others. According to semiotic theory though, objects can signify in non-arbitrary ways. It is, in fact, the primacy of physical modes of signification, such as iconicity and indexicality, that scaffolds the emergence of symbolic ways of meaning-making. As will be proposed, it is through a process of semiotic scaffolding that symbolism and selfhood both developed. Rather than taking the human self for granted, we can see it as emerging along with increasingly arbitrary visual signs. Seeing how the process of our cognitive becoming must have been catalysed by material signification, we will be ultimately arriving at the conclusion that cognitive archaeology might have something to gain from meeting with visual semiotics.
In making these claims though, evolutionary cognitive archaeologists seem to rely on a pair of misguided assumptions: (i) that material signs are inherently symbolic because they are all arbitrarily linked to their meaning; and (ii) that the use of symbolic culture implies human beings with a concrete sense of self, which employed material signs in order to communicate personal information to others. According to semiotic theory though, objects can signify in non-arbitrary ways. It is, in fact, the primacy of physical modes of signification, such as iconicity and indexicality, that scaffolds the emergence of symbolic ways of meaning-making. As will be proposed, it is through a process of semiotic scaffolding that symbolism and selfhood both developed. Rather than taking the human self for granted, we can see it as emerging along with increasingly arbitrary visual signs. Seeing how the process of our cognitive becoming must have been catalysed by material signification, we will be ultimately arriving at the conclusion that cognitive archaeology might have something to gain from meeting with visual semiotics.
John M. Kennedy: From dots to the sacred: Theory of visual and tactile pictures
What is the scope of pictures? Why? And what is missing? Find five flaws! Theory of vision, touch and visible pictures is incomplete. One, it is challenged by the effectiveness of tactile pictures for the blind. Two, consider the Peter Coppin question: Why do tangible and visible dots suggest lines but lines do not suggest dots? Going beyond Wertheimer, a “dots trigger equations” theory is helpful, but often just an allusion. Three, even with less than adequate information, outlines suggest foreground and background. Why so flexible? Perhaps familiarity can help. But outline does not suggest chiaroscuro. Why so inflexible? Perhaps familiarity cannot help! Four, linear perspective works well. It creates illusions in reading picture surfaces. But it has dramatic limits. We track azimuth better than elevation. Hence we fail to perceive lines converging to distance points as parallel. Five, a picture with a trope has a feature that demands an explanation. These range freely over metaphysics, covering the ontologies from the mundane to the holy. But theory of the acquisition of metaphoric pictures, in individuals and evolution, is in its infancy -- literally and metaphorically. What metaphoric pictures might infants, apes and parrots understand? Let us speculate. Question and exclamation marks, conventions for surprise, should fail. Extended, curved, bent and wavy lines for fast, steady, jerky and wobbly motion -- alongside blur, trailing lines, and the common fare of comics – should succeed. What about radiating lines for pain? For shouts? For fiery tastes? Yes, surely these too.
Jean-Marie Klinkenberg: Visual experience and semiogenesis
Half a century ago, one of the masters of semiotics observed: "Without knowing anything more about the nature of meaning, we have learned to better know where it manifests itself and how it shifts" (Greimas, 1970: 17). Semiotics and linguistics have chosen to ignore the problem of the nature of meaning, or have decided to postpone it for later, thus depriving themselves of knowing how and why meaning is born.
In my lecture, I will show that meaning emerges from experience, at its most basic level: sensory experience. This theme will be exemplified with visual perception, which we have explored in our earlier work and especially in our Treatise on the visual sign (Group μ, 1992). But the model that will be described here will mutatis mutandis be valid for perceptions transiting through other sensory channels.
I will first show how our peripheral organs receive and organize the stimuli coming from the outside world by comparing them, initially two by two, and radicalizing their differences. This is the important principle of the dipole. This principle enlightens the oppositional conception of meaning, which is a postulate of structuralism. The product, already very sophisticated, of these sequences of treatments is in turn treated by a logic analyser, which plays the role of interpreter of the phenomena. This is a first moment in the experiential process of meaning formation, or anasemiosis.
It should be noted that in this process of anasemiosis, there is a grouping of experiential data. This grouping, or articulation, occurs for purposes of economy (uniting), but each time in distinct spaces, as well as for purposes of efficiency (dividing): this is the fundamental process in the constitution of meaning.
A semiotic theory can be complete only if it accounts for this anasemiotic process, but also if, on the other hand, it is supplemented by a pragmatic dimension. In other words, it must show how meaning can return to the world in the form of action, a process we will call catasemiosis.
In my lecture, I will show that meaning emerges from experience, at its most basic level: sensory experience. This theme will be exemplified with visual perception, which we have explored in our earlier work and especially in our Treatise on the visual sign (Group μ, 1992). But the model that will be described here will mutatis mutandis be valid for perceptions transiting through other sensory channels.
I will first show how our peripheral organs receive and organize the stimuli coming from the outside world by comparing them, initially two by two, and radicalizing their differences. This is the important principle of the dipole. This principle enlightens the oppositional conception of meaning, which is a postulate of structuralism. The product, already very sophisticated, of these sequences of treatments is in turn treated by a logic analyser, which plays the role of interpreter of the phenomena. This is a first moment in the experiential process of meaning formation, or anasemiosis.
It should be noted that in this process of anasemiosis, there is a grouping of experiential data. This grouping, or articulation, occurs for purposes of economy (uniting), but each time in distinct spaces, as well as for purposes of efficiency (dividing): this is the fundamental process in the constitution of meaning.
A semiotic theory can be complete only if it accounts for this anasemiotic process, but also if, on the other hand, it is supplemented by a pragmatic dimension. In other words, it must show how meaning can return to the world in the form of action, a process we will call catasemiosis.
Douglas Niño: Time, Attention, and Visuality in Meaning-Making
Traditionally, the attempts to explain the meaning of visual signs are based on the association of expression and content, by establishing codified systems which are, in turn, understood in a Saussure-Hjelmslev-like kind of extrapolation, imported from Linguistics. Even Peirce’s iconism was understood in a similar fashion. Frequently, the considerations on visual signs became “akinetopsic”, since their capacity to transform themselves was not glossed. As a consequence, the analyses based on this approach yield systematic results for phenomena that are rather generally dynamic. With the advent of cognitive approaches, the explanation of meaning does not revolve around codification systems, but around the agent’s cognitive capabilities. This, however, involves on the one hand a commitment with the empirical findings of the cognitive sciences and, on the other, a grasp of the conditions under which meaning arises in the human experience. In this sense, important theoretical and methodological consequences for visual semiotics may follow from incorporating, in the explanation of meaning-making, attentional and temporal conditions. Firstly, bringing forth attention (particularly visual attention, both in its bottom-up and top-down dimensions) requires scrutiny of the field, where the meaning-making will depend no only on the figure/ground gestaltic configuration, but crucially on a structure resulting of visual enaction and determinable saliencies (spacial dimensions, frameworks of affective valences, etc.). Visuality will not be about mere visual items, but about visual items entangled within the whole of experience. Similarly, the problem arises of whether a visual item is considered a sort of representation or not and, if so, of what kind. The cognitive effects of making sense of something as not representational are not certainly the same as consider it as an icon, an index or a symbol. Secondly, the temporal dimension has effects no only on that which is observed (the exposure time, for example, has an effect on meaning-making, as horror movies filmmakers know all too well), for the very process of meaning-making requires an internal time. In this talk, I will expound the conditions under which these temporal and attentional conditions help to configure the experience of meaning-making, as well as some of its theoretical and methodological consequences for a visual semiotics cognitively oriented.