Multiform References (MFRs) are the rhetorical devices that authors use to shift their readers’ attention between words and images to create a new, unified representation or meaning.
I argue here that MFRs are an intimate rhetorical device and that, consequently, they have a significant capacity to evoke emotions among both the author and the reader (if we focus on illustrated texts). What is in MFRs that make them so?
Firstly, the choice of which words and images to use in a verbal-visual sequence seems to be more revealing than the choice of which words to use in an only-verbal one or which images in an only-visual one. The use of two different semiotic systems in one message, on the one hand, and the lack of guidance as to how to use multiform rhetoric, on the other, open room for creativity and improvisation. These, in turn, shed light on rhetorical decisions based on personal sensitivities and inclinations.
Noa Yaari, Your Stats (detail of “Artist in Residence“), 2020. Ink and acrylic on paper. 23 x 30 cm. Toronto.
Secondly, MFRs are the space and time (or spacetime) between words and images. As such, they provide the readers with the opportunity to integrate verbal and visual signs into a new unified one. This process of integration touches upon associations that we established at a young age, mostly when we acquired language. When our parents pointed something and told us how it is called and did so repeatedly, they created to and for us a system with which we can represent the world. When we point an image in our text and claim something about it, we parent our readers, and arguably, ourselves.
Noa Yaari - The Multiform Grammar Lab 