Multiform References

The integration of the verbal and visual components of multiform arguments (MFAs) into a cohesive utterance is possible by juxtaposing and entangling the semantic and visual aspects of words and images. This is feasible since printed words and printed images have the capacity to be both meaningful and visible at the same time. Further, their visibility and meaningfulness can very well exist while the other medium is also visible and meaningful. The mechanisms that coordinate and integrate the visibility and meaningfulness of words and images within an MFA are “multiform references” (MFRs).

Noa Yaari, Communication, 2018.

Since MFRs are a kind of reference, they have a potential to influence the readers to shift their attention from words to images and vice versa; however, there is nothing intrinsic in MFRs that guarantees a change in readers’ behavior. Thus, when we discuss MFRs, we relate to a rhetorical device that has a potential to change readers’ mode of reading and observing the text. The actual power of MFRs to change readers’ behavior will be clearer after we track readers’ eye movements while they consume MFAs. Tracking readers’ eye movements will help us reveal the varied patterns involved in reading illustrated texts, among them the elements that drive readers to shift their attention between the verbal and visual components; the velocity of readers’ reaction to visual and semantic stimuli (that is manifested in rapid eye movements, i.e., saccades, that are measured in milliseconds); and the duration in which their gaze rests upon the different components. Finally, tracking readers’ eye movements will also allow us to understand better the “compelling” element that thinkers have ascribed to word and image relation.

 

Noa Yaari, Business, 2018.

Indeed, Saussure, Wittgenstein and Bal have pointed out the power embedded in word and image relation to forcefully evoke thoughts in one’s mind. Charles Sanders Peirce’s classification of signs into three modes: symbolic, iconic and indexical, may be considered as a framework to think about the compelling or involuntary effect word and image relation has upon readers. According to Peirce, indexical signs are connected with the things or objects they represent by a physical, organic connection (2.229). Therefore, the direct and contiguous connection between indexical signs and the things they represent enables indices to “direct the attention to their objects by blind compulsion” (2.306). Peirce also argues that indices can be thought of “as a fragment torn away from the object, the two in their existence being one whole or a part of such whole“ (2.230). If words and images have pushing and pulling forces between them, that can shift individuals’ attention from the words to the images and vice versa, across the MFA’s space, it implies that words and images may be physical parts of a whole. This conceptual framework suits well my understanding of MFAs and especially MFRs, that might be a whole whose verbal and visual poles are “torn” from its body.

 

Bal, Mieke. Reading “Rembrandt”: Beyond the Word-Image Opposition. Cambridge: Cambridge University Press, 1991.
de Saussure, Ferdinand. Course in General Linguistics. Edited by Charles Bally, Albert Sechehaye and Albert Riedlinger. Translated and annotated by Roy Harri. London: Duckworth, 1983.
Peirce, Charles S. Collected Papers of Charles Sanders Peirce, 8 vols. Edited by Charles Hartshorne and Paul Weiss, Cambridge, MA: The Belknap Press of Harvard University Press, 1931-58.
Wittgenstein, Ludwig. Philosophical Investigations. Translated by G.E.M. Anscombe, P.M.S. Hacker, and Joachim Schulte. Malden: Wiley-Blackwell, 2009.
Posted in Uncategorized

Multiformity vs. Multimodality

The concept “multiformity” resonates with the concept “multimodality,” not only in the way it looks and sounds, but also the broad social and cultural contexts from which it develops and to which it is addressed. Both concepts allow us to explore forms of communication that mix what we apprehend as distinct channels of transmitting information: written and spoken words; still and moving visual images; sound; gestures and so on. Both concepts ponder the great influence new technologies have on human interaction, and their implications in the arts and education, especially in the digital era. Therefore, it is important to clarify the main difference between the “multiformity” that I offer and Gunther Kress’s “multimodality” (2010).

The difference between the two concepts or approaches lies in the weight the two ascribe to social versus biological factors in human communication, and consequently the approaches’ use of the concepts “mode” versus “form.” “Mode,” according to Kress, is a semiotic resource for making meaning; its potential is in the affordance of the material it is made of (i.e. sound in speech or graphic stuff in writing), while its realization is the practical use of materialistic potentials by members of a given society. Further, modes differ in their underlying “semiotic logic;” the organizing principle under which they are conveyed. Kress claims, for example, that the logic of words is to follow each other in temporal sequence, and the logic of images is to display their elements simultaneously in space (2010: 81-82).

Noa Yaari, Multimodality, 2014.

Kress’s approach to multimodality is social; it sees the sign-makers’ use of signs as the mechanism that generates potential semantic meanings, rather than grammar, that can be thought of as an abstract and fixed system of rules originating in the brain. According to the social approach, the process that refashions lingual resources and practices is humans’ motivation to frame meanings in social context; this enables lingual interactions to reflect the present, with its instability: its social and technological transformations (Kress 1996, 2010).

Noa Yaari, Multiformity, 2018.

“Form,” on the other hand, relates to the most fundamental level that one can explore in relation to verbal and visual, or any other type of communication, while the level itself is defined according to the question in hand. For example, while exploring written language, we may focus on forms such as a graphic element in a letter, a letter, a sentence, a shape of a paragraph, etc. Forms in spoken language can be a phoneme, a word, a whole speech, and even a sound in the background of the speech. Forms in visual (non-verbal) communication can be a brush stroke, an artwork, a garden or building, but also a graphic element within a letter or a gesture of a speaker, during their speech.

Thus, the use of the concepts “form” and “multiform” attempts to open a theoretical framework to immediate and direct semiotic phenomena of any type and scale, so that they can shape and convey meanings synergistically, before any established logic interferes between them. It implies that “forms” function in high velocity; they do not have time to distinguish between temporal sequence, on the one hand, and simultaneous display in space, on the other, nor between ”time” and “space.” The reason why they are fast is that they function on both the social and biological levels. The body’s reaction to any kind of stimuli and processing them into meanings, given probable social conditioning and contexts, cannot rule out survival mechanisms.

 

Kress, Gunther. Multimodality: A Social Semiotic Approach to Contemporary Communication. New York: Routledge, 2010.
Kress, Gunther, and Theo van Leeuwen. Reading Images: The Grammar of Visual Design. New York: Routledge, 1996.

 

Posted in Dissertation, My Art

What are Multiform Arguments?

Multiform arguments (hereafter MFAs) are arguments that are composed of words and images. MFAs claim that a certain phenomenon has happened or explain the reasons for its occurrence. I became aware of MFAs while reading illustrated historiography, especially of cultural history. I noticed that when I wanted to point out how cultural historians use visual evidence in their publications, I lacked terms to describe what I read and observed. While it was easy to recognize and pinpoint textual units such as “words,” “images,” “sentences,” “captions” and ”list of illustrations” in those publications, it was challenging to describe combinations of words and images in the same argumentative sequence and space.

Noa Yaari, This is My Home, 2018. Toronto

The lack of terms that denote multiform components in historiography, I believe, has deterred authors and readers from exploring these hybrid forms, and subsequently constructing them more sophisticatedly, creatively and effectively. Moreover, since studies have shown that language shapes thinking, it would be reasonable to assume that without copious nomenclature not only the language of illustrated arguments might stay restricted, but also the notions this language is meant to convey. In addition, without adequate nomenclature it is impossible to establish a critical discourse about using multiformity in historical arguments. The concept “MFA” seems to be a proper beginning of a solution, since it sets forth the multiplicity of the semiotic systems and their combination within an argument, as well as the argumentative context in which these combinations function. Moreover, the concept “MFA” allows us to investigate the semantic, spatial and epistemological aspects of verbal-visual insights; to examine their cohesiveness, focusing on the way they expand in space and the time that it takes to consume their body, and their coherence, paying attention to their logical consistency. “MFA,” therefore, promotes the creation and communication of historical knowledge through diverse and hybrid forms of thought and expression. It cultivates an experimental, theoretical and critical approach to textual multiformity; and another perspective on life, in the past and right now.

Posted in Dissertation, My Art

History Sells

            

Noa Yaari, History Sells (homage to Andy Worhol), 2015. A bookmark. 20.5 x 5 cm. Toronto
Posted in My Art

In the

Noa Yaari,  I Lose Control, 2018. Ink on paper. 21 x 27 cm. Toronto

 

Image source – Sandro Botticelli’s The Birth of Venus (1482-1485):
https://en.wikipedia.org/wiki/The_Birth_of_Venus#/media/File:Venus_botticelli_detail.jpg
Posted in My Art

What Makes Visual Evidence “Evidence”?

Alfa Romeo 4C Spider

In April’s post, I asked if – within the industry of creating and communicating historical knowledge – there is any epistemological significance to visual evidence that was photographed by the historian who writes about it. I claimed that participating in the production of the visual evidence is an “opportunity to compare the production of the artwork to the production of its reproduction;” and that, based on this comparison, historians may become more aware of the dependence of the visual evidence on the context in which it is produced, seen and recorded, and consequently to its limitation in representing broader cultural and social phenomena.

Ferrari 488 GTB

So, if producing visual evidence in illustrated historiography is no different than artistic activity, as both processes are deeply influenced by the ever-changing conditions around them, what makes the visual evidence “evidence”? What is in an object or image that makes it a source of historical knowledge, within and outside of the publication of the historical research? If we imagine an artwork and illustrated-history book as two artworks or two pieces of evidence, that exist at the same time, and in either case the illustrated book reproduces the artwork as an indication of a broader phenomenon, then the book has to “freeze” the artwork and itself at a certain point in time. It must do so to claim that what it creates and communicates is “knowledge.” What power does the book have over the artwork if they are two simulations artworks or pieces of evidence? And whatever the answer may be, does the artwork have the same or similar power over the book?

Lamborghini Huracán Performante Spyder

The Lamborghini above is driving left to the white line. Also the Alfa Romeo. What does it mean? Does is have to mean anything?

 

Alfa Romeo 4C Spider – image source: http://www.carmag.co.za/car-reviews/road-tests/road-test-alfa-romeo-4c-spider/
Ferrari 488 GTB – image source: https://auto.ferrari.com/en_EN/sports-cars-models/car-range/ferrari-488-gtb/
Lamborghini Huracán Performante Spyder – image source: https://www.lamborghini.com/en-en/models/huracan/huracan-performante-spyder
Posted in Contemplations

Historians Photograph their Visual Evidence

The visual material in illustrated historiography usually results from a chain of practices, most noticeable are the artistic creation, the photographing of the artwork, and the printing of that photograph in the book. This chain of practices is the industry of creating and communicating historical knowledge. Within that industry, is there any epistemological significance to visual evidence that was photographed by the historian who writes about it?

Women on Motorcycles I

When we see the historian receiving the credit for the photo, we assume that they went to the place in which the artwork was, to photograph it; observed the artwork directly; operated a camera to photograph the artwork; gave the recording data to the publisher to print it in the book; informed the publisher that they are the creators and owners of the recording material; and, finally, observed the reproduction of the artwork in their book. Each of these practices is an opportunity to compare the production of the artwork to the production of its reproduction. For example, the attempt to produce a high-quality photograph of the artwork requires the photographer to explore the lighting conditions of the artwork. This exploration can evoke empathy with the creator and contemporary observers of the artwork, since they, like the photographer, needed light to see it, let alone its details.

Women on Motorcycles II

When historians are involved in the production of the visual evidence, they become more aware of the epistemological challenge of using visual evidence to make claims about a broader cultural and social phenomenon. The unmediated experience with the artwork, that can only happen when the historian and the artwork are in the same space, demonstrates to the historian that the context in which the artwork has been created, displayed and seen, in fact construct the artwork. There is nothing essential in the artwork that cannot be influenced by the conditions around it. This is exactly why we can explore the past through analyzing artworks, and this is also why we have to remember that artworks are limited sources, as they change with time.

Women on Motorcycles III

This insight positions visual evidence in an epistemological context that includes, among other factors, the ever-changing technologies of recording and disseminating visual data. Historians who photograph their visual evidence ought to be more aware of this epistemological relativity; the practical participation in producing visual evidence shows that the status of the evidence as such depends on many and varied factors that are not inherent to the artwork. Whether historians use this notion as part of their own work is another question. We do know that it can advance our understanding of the complexity of studying the past, and the visual material it is constantly leaving behind.

 

Women Motorcycles I: image source – http://www.youmotorcycle.com/women-in-vintage-motorcycle-photography.html
Women Motorcycles II: image source – http://longicaudal35.rssing.com/chan-18197915/all_p6.html
Women Motorcycle III: image source – https://rideapart.com/articles/10-reasons-to-date-a-woman-who-rides-a-motorcycle
Posted in Contemplations, Dissertation

Renaissance Society of America – New Orleans, March 22-24, 2018

At the RSA 2018, I commented on three papers that art historians Dr. Jorge Sebastián LozanoDr. Víctor Mínguez Cornelles and Dr. Inmaculada Rodríguez Moya presented in the session: “Between Word and Image: Verbal-Visual Representations of Sixteenth- and Seventeenth-Century Spanish Royal Women,” which I organized, and Dr. Julie Campbell chaired.

This is what I said:

Over the last few weeks, I’ve been watching videos on Youtube about twin-flames; these are souls that were once one, that have separated, and are on a journey to reunite. It can take forever, but it eventually happens, if they are patient, allowing God’s plan to work out. When they unite, the videos say, they make the world a better place; but, there are some issues and suggestions that we should all be aware of, when it comes to twin-flames, and that the videos address as well. For example, what to do when your twin-flame is married to someone else (oops), what to do when your twin-flame ignores you, and how to deal with the great emotional intensity when the unbreakable reunion finally occurs.

Noa Yaari, I Love You, 2018

Word and image relation is not so different from twin-flame connection; it, too, manifests varied measures of similarity, distance, attraction, and the potential to create something whole with an everlasting quality. The three papers that we have just heard tell us a story about the use of word-and-image relation to establish status and legacy for three 16th and 17th century Spanish royal women. These papers show that throughout history, people understood how the human mind reacts to combinations of words and images. We irresistibly – probably unconsciously – accept verbal-visual constellations as accurate representations of whatever they stand for. And while we are wired to do so, what determines the form of these hybrid representations is the rise of specific needs and availability of verbal and visual resources.

Noa Yaari, I Love You, 2018

According to Jorge’s paper, when the beauty of Empress Isabella became a necessity, it wasn’t essential to have her around, nor any of the first-hand testimonies of her appearance. Ironically, as Jorge points out, the instillation of the Empress’s beauty in the collective memory was due to the narratives on St. Francis of Borgia, who allegedly converted, traumatized, after seeing his patron’s corpse. Painting beautiful portraits of the Empress, on the one hand, and conveying how devastating it was to see her losing that beauty, on the other, created an immortal image of a mortal woman.

Noa Yaari, I Love You, 2018

As Victor’s paper shows, Queen Isabella of Valois became a symbol of a new peaceful and powerful era in Spanish history; her signification as such was possible through her marriage to Philip II, but more importantly, through crossing different semiotic systems in relation to her. The juxtaposition of the motto “IAM FELICTER OMNIA” (“Everything now felicitously”), an image of a sun (with everything that it symbolizes) and an image of a moon (with its own symbolism) creates a representation of a new situation in global politics. A comparison between the 1559 and the 1560 emblems reveals the crucial role of the spatial-relation between words and images in generating a specific meaning; in the later emblem, we see the motto free from its ribbon and closer to the two celestial bodies, which are much more similar now, signifying that politics have become more felicitous not only on the global level, but also between the “sun” and the “moon.” As Victor suggests, the later emblem radiates to the world that Spain is in the hands of a strong ruling couple.

Noa Yaari, I Love You, 2018

Finally, Inma’s paper demonstrates the complex connection between one’s life and death and their verbal-visual representations. For example, before Isabella Clara Eugenia died in 1633, she requested to be buried in a specific chapel, in a particular church with a monument and double-tomb for herself and her cousin; however, her request was never entirely fulfilled, as her gravestone is, in fact, relatively modest and simple. As Inma points out, by using verbal and visual rhetorical devices, Puget dele Serre made his illustrated eulogy of Isabella her mausoleum and portrait. In this case, we could argue, one doesn’t sit in but with the place to contemplate the deceased’s spiritual and political virtues. More generally, perhaps in addition to the various kinds of rhetoric that are used in illustrated books, the fact that the books require direct touch and activity from the readers fosters the memory of their content. At any rate, the three papers that we have heard today indicate that 16th and 17th century Spanish culture had found a way to ensure that their royal women’s legacy will burn forever. The fact that we are discussing them today suggests that so far so good.

Noa Yaari, I Love You, 2018

 

Posted in Contemplations, My Art

Drawing Childhood Memories on Money as a Teaching Tool

The following article was published in Innovatus, York University’s monthly pedagogical newsletter, on January 19, 2018. It describes an art exercise I gave in my tutorials.

In the last tutorial of the fall term, students in the History course Making Money drew their early memories on money. This exercise, given by artist and Teaching Assistant Noa Yaari, builds on two writing assignments Course Director Professor David Koffman gave that term. The first is to write about an early childhood memory on money, and the second is to analyze an image in which money plays a central role.

“Drawing a memory requires the students to coordinate the image in their mind, their body, art supplies and the work that they gradually create” says Yaari. “It’s challenging and important, since while we are developing our analytical verbal skills, we tend to forget the role our body and senses have in creating and communicating knowledge. We also neglect them as a teaching and learning tool.”

Students drawing childhood memories on money, in History Course “Making Money”

According to Yaari, the distinction between “body” and “mind” limits our perception of ourselves as learning and teaching beings. “I find teaching to be a great opportunity to explore learning processes,” says Yaari. “For example, returning to a memory that we already wrote a paper on to physically draw it is to literally feel the varied ways in which we can approach the same topic. Welcoming this diversity enriches our understanding of each of the approaches, enabling us to combine them and offer new ones.”

Artwork by Maria Toufiq

“Since money is anyway a sensitive issue, approaching it from different angles helps understand the complexity of learning about it. I hope that this exercise will have a long-term impact on the students,” adds Yaari.

Artwork by Lucas Amello

Student Lucas Amello worked on his memory on selling lemonade in front of his family’s house, to purchase a desired bicycle. He shares his experience: “I think there is a monumental difference between the articulation and depiction of a memory. I find that I attain a greater level of clarity after partaking in an artistic exercise such as this. I was surprised at how clearly I could summarize my 1200-word paper. In the short span of 15 minutes, I was able to effectively depict the full extent of my childhood experience, showcasing in a series of pictures the extensive impact it had on the way I view currency, and the manner in which I conduct my finances.”

Artwork by Narjis Panjwani

Professor David Koffman reflects on the classroom activity. “I was very pleased by Noa’s initiative to have students use art to thicken their experiences of the material we analyzed in class,” says Koffman. “As a General Education course, Making Money is designed not just to promote thinking about money from a variety of new perspectives, but to encourage careful questioning about how different fields of scholarship know what they know. This in-tutorial art exercise took the course aim in an entirely new direction: by returning to the same memory, students got the chance to consider how they can come to know differently what they already know.”

A PhD candidate in the Faculty of Graduate Studies, Yaari, who is finishing her dissertation in History, writes on how historians use visual evidence in their publications: “I have no doubt that my research, teaching and art have merged in this exercise. I feel privileged to have these students, course director and the Department of History, who kindly purchased the art supplies, to experience new pedagogical adventures.”

A Flickr gallery showing the students’ work: https://www.flickr.com/photos/yfile/25893573088/in/album-72157692384397525
The post in York University’s Innovatushttp://yfile.news.yorku.ca/2018/01/18/drawing-childhood-memories-on-money/
Posted in My Art, Teaching

Self Portrait, 2000-2018

Noa Yaari, Self Portrait, 2000-2018. Mixed media, Tel Aviv-Toronto
Posted in Dissertation, My Art