February 8, 2012
Chapter 1

Chapter 1: Multimodal Literacy


“odds and ends”

(Levi-Strauss, C. 1966. p.35)

With a world of ever changing languages, newly evolving acronyms and more signs than most people can even begin to relate to, there is an ever-growing necessity to read images. This is not a new phenomenon, but the way it is approached has changed through new-media

While many linguists[1] would love a chance to get rid of the emoticon (or smiley) from the screens of their phones and laptops, it has created a generation of children who can read image and text as one. Where the main obstacle was, at first, found through the way we read concrete poetry as two separate objects (the text and the image) that is changing, and image and text are being given a chance to interact with each other in a very new way.

One of the first modern examples of the deliberate integration of signs and symbols into everyday reading was Otto Neurath’s work with Isotypes (1925-34) (fig 5), later developed into the globally recognized public signage project by Jock Kinnear and Margaret Calvert (fig 6). This use of a signifier to convey what would have needed paragraphs of writing to explain was a huge leap forward in to a largely unexplored world of dual literacy. It did not however start the current revolution into a generation of multimodal readers and writers. Described here is the current state of reading, by Traci Gardner:

“Today’s media-savvy students compose and read texts that include alphabetic and character-based print, still images, video, and sound. They listen to pod-casts, watch animations on the Internet, film their own videos, and compose visual arguments on paper and online. These rich, multilayered texts demand multimodal literacy skills of their readers, who must navigate the different, intersecting media. These media can include visual, auditory, textual, and spatial ways of making meaning.” (Gardener, T. 2007. p.93)

 The necessity that new-media has created is one of dissection. The dissecting of all the information put in front of you to make a whole, especially online. Hyperlinks were very quickly taken up by a global audience as the new library, but are now evolving into a crucial element of online information. Where before, a hyperlink was the equivalent of a footnote, giving the reader the option to look up further information, it is now, often, a link to a separate extract of the same text, transferring knowledge between two categories. If Flikr, YouTube, SoundCloud[2], are a more relevant way of portraying an idea, then why write it down, when a link to an external page can be far more relevant than reading about it.

 These rich, multilayered texts demand multimodal literacy skills of their readers, who must navigate the different, intersecting media. These media can include visual, auditory, textual, and spatial ways of making meaning.” (Gardener, T. 2007. p.93)

 An unprecedented vibrancy has been produced out of this now and is still growing. Traci Gardener’s views on these “multimodal literacy skills” (Gardener, T. 2007. p.93) are one of few academic voices that praise the effect that new-media-language is having on the global culture. Most take a more concerned view that there is an assumed knowledge of how to read these new language tools[3].

While this is valid, it is probably a concern that will fizzle out. These new language tools can be hugely beneficial, and are learned through emersion. Emersion is generally seen to be the absolute best way to learn a language where it is possible. TEFL Express (TEFL, 2012) take the view that to learn a language well, there must be a crucial emphasis put on varying the states of learning. While they focus on teaching correct, conversational English to non-English speakers of a varying age range, they still hold the view that interaction with a language is far better than it being conventionally taught. If it becomes the case, through this mass emergence of new technology and the language that goes with it, that everyone has the ability to use the most basic of web tools; such as a hyperlink or basic colloquial acronyms (LOL – Laugh Out Loud, OMG – Oh My God[4]) then there should be no arguments over these evolving languages. Especially where they improve a general social literacy in a world where literacy levels are declining.

 Norway, the home of Nokia[5], has a thriving cell phone culture, portrayed in Gerard Goggin’s aptly named ‘Cell Phone Culture’ (2006). This book tells the tale of SMS[6] history and of what it has become. Repeated references are made to Trosby’s, ‘SMS: The Strange Duckling of GSM’ (2004), where parallels are drawn to an old Norwegian fairytale character Askeladden “who picks up all kinds of items that he encounters given the presumption that it may come to use some day.”  Askeladden is made out to be something of a bricoleur, making everything count. This attitude towards his adventures reflects that sense of evolution in language. Bits and pieces are picked up, and put back together again. Words and images combine to create a knowledge base, or a story.

Claude Levi-Strauss tries to sum up the concept of bricolage as a combination of “odds and ends” (1966, p.35). The ability to put these odds and ends together again was always something attributed to those working creatively. Now though, that piecing together is becoming a crucial part of reading in any field, as this sense of a multimodal language forms.

Georges Perec writes throughout his verse in ‘Species of Spaces’ (1974) in a way which demands the page to be laid out in a specific manner, whereby the page itself becomes as important, if not more important, than the text itself. The openning extract has been included here as an image (fig 7) to express further this cross-over between word and image. The words themselves, rather than the sentences they make up are often, in Perecs view, the most cruical, meaningful part of texts, in the same way that W. J. T. Mitchell talks about in his review of language; ‘Word and Image’ (1996). His consideration of what words are, written words specifically, concludes that all text is nothing more than

            “black marks on a white background”. (Mitchell, W. J. T. 1996. P.51)

They may seek to express the spoken elements of language, and to ultimately express the visual devices which language seeks to convey, but in the end, a letter is just a black mark, a shape like any other. If, in this multimodal idea of modern literacy through new-media, there is a sense of more accurately linking signifiers to their signified then these marks should be interchangable to cope with changes. Language is always evolving, so it is therefore only right that it’s written counterpart follows suit.


[1] See Lynne Truss’ ‘Eats, Shoots and Leaves’ (2003) for an in depth analysis of modern punctuation.

[2] Flickr: image sharing; YouTube: video sharing; SoundCloud: audio sharing.

[3] “hypertext is no longer as much a puzzle as it was in 1998 (now that it is naturalized enough that your familiarity with it and ability to think your way to connections between and among sometimes disparate texts is simply assumed as you browse online)” (Wysocki, A.F. 2007. p.282-283)

[4] Acronyms commonly used in texting and online discussions were added to the English Dictionary in 2009 (Oxford, 2009)

[5] Nokia (1865): one of the first and largest commercial mobile phone manufacturers.

[6] Short Message Service.