I recently started to work at an online project that teaches English-speaking kids another language. The program encourages them to shift their attention repetitively between the written words and the images to acquire the new language. These shifts integrate the words and the images into a new hybrid sign that includes both the name of the object it represents and its appearance.
These shifts are what I termed as “multiform references” (MFRs), that push and pull the attention between the words and the images that share a field of vision. MFRs have a crucial influence on the leaners’ apprehension and comprehension of the world, as well as their development of ethical values. For example, MFRs may write on their verbal end “mother” or “father,” and on their visual one, portray a person in a specific activity as if it is an inherent part of the learned term.
Imagine an application that indicates where authors use MFRs to subtly express their ideologies, ideas about identities and their appropriate roles in society, and right and wrong. What kind of artificial intelligence would it need?
Noa Yaari - The Multiform Grammar Lab 